Crawling APIs Using Neo4j - Building The Star Wars Graph
William Lyon / December 14, 2015
1 min read •
A common task when working with data from APIs is crawling the API and inserting the results in a database. In this example we'll import data into Neo4j by crawling a public API, building a graph in Neo4j as we go along. We'll use Neo4j as a queuing mechanism to store URLs for placeholder resources waiting to be fetched.
Specifically, we'll use SWAPI.co to build a graph of information from the Star Wars universe. SWAPI contains data about characters, planets, starships (and much more!) that appear in the Star Wars universe. Much of the data is collected from Wookieepedia.
The Data Model#
The graph data model we'll use when building the graph.
To see the full example please check out the iPython Notebook here.
Subscribe To Will's Newsletter
Want to know when the next blog post is published? Subscribe now!