A complete guide to deploying Neo4j Enterprise 5.x on any VPS (Digital Ocean, AWS, GCP, Azure, or on-premises) with SSL/TLS encryption, database restoration, and read-only user configuration.

Deploying Single Instance Neo4j On A VPS With Custom Domain

Building interactive graph visualizations with Sigma React - from basic setup to advanced customizations

A Look At Graph Visualization With Sigma React


import Image from 'next/image';
import Subscribe from '../../components/Subscribe';

Every year with my family I head out into a nearby National Forest with a saw (except when I forget to bring it) and a National Forest Motor Vehicle User Map (MVUM) to find the annual Lyon family Christmas tree. I always wondered if I could analyze the data for the MVUMs to make my own map for hunting Christmas trees - so that's what I did for this map as part of the [30 Day Map Challenge.](https://github.com/johnymontana/30-day-map-challenge/tree/main)

I also did a bit of research using aerial imagery and QGIS to find areas that might have the perfect Christmas trees, then loaded the National Forest MVUM data in [Wherobots Cloud](https://wherobots.com/wherobots-cloud/) to find the forest service roads leading to the areas I annotated in QGIS with Spatial SQL.

<Image
  src={'/static/images/finding-tree/07-navigation-final.png'}
  alt={'Finding the perfect Christmas tree'}
  width={3300}
  height={2550}
/>

To follow along create a free account in [Wherobots Cloud](https://www.wherobots.services)

<Subscribe />

## Loading National Forest Motor Vehicle User Map Data

The US Forest Service publishes the National Forest Motor Vehicle User Map (MVUM) which shows roads and trails for accessing National Forest land. These maps are usually my primary navigation method when I'm driving into a National Forest. The USFS makes this map data available for download [here](https://data-usfs.hub.arcgis.com/datasets/motor-vehicle-use-map-roads-feature-layer/explore) as GeoJSON, CSV, or Shapefile.

I downloaded the Shapefile format, then uploaded it to Wherobots Cloud via the "Files" tab, which allows me to access this data via S3.

<Image
  src={'/static/images/finding-tree/files1.png'}
  alt={'Working with files in Wherobots Cloud'}
  width={3134}
  height={1700}
/>

Now we're ready to import that Shapefile into SedonaDB.

```py
# To import a Shapefile into SedonaDB we can use the built-in ShapefileReader which will
#   give us a SpatialRDD then convert that SpatialRDD to a Spatial DataFrame
# Then we create a new view called roads with this MVUM map data
# Finally we print the schema of the roads_df DataFrame

spatialRDD = ShapefileReader.readToGeometryRDD(sedona, S3_URL_ROADS_SHP)
roads_df = Adapter.toDf(spatialRDD, sedona)
roads_df.createOrReplaceTempView("roads")
roads_df.printSchema()
```

```md
root
|-- geometry: geometry (nullable = true)
|-- OBJECTID: string (nullable = true)
|-- RTE*CN: string (nullable = true)
|-- ID: string (nullable = true)
|-- NAME: string (nullable = true)
|-- BMP: string (nullable = true)
|-- EMP: string (nullable = true)
|-- SEG_LENGTH: string (nullable = true)
|-- GIS_MILES: string (nullable = true)
|-- SYMBOL: string (nullable = true)
|-- MVUM_SYMBO: string (nullable = true)
|-- JURISDICTI: string (nullable = true)
|-- OPERATIONA: string (nullable = true)
|-- SURFACETYP: string (nullable = true)
|-- SYSTEM: string (nullable = true)
|-- SEASONAL: string (nullable = true)
|-- PASSENGERV: string (nullable = true)
|-- PASSENGE_1: string (nullable = true)
|-- HIGHCLEARA: string (nullable = true)
|-- HIGHCLEA_1: string (nullable = true)
|-- TRUCK: string (nullable = true)
|-- TRUCK_DATE: string (nullable = true)
|-- BUS: string (nullable = true)
|-- BUS_DATESO: string (nullable = true)
|-- MOTORHOME: string (nullable = true)
|-- MOTORHOME*: string (nullable = true)
|-- FOURWD*GT5: string (nullable = true)
|-- FOURWD_G_1: string (nullable = true)
|-- TWOWD_GT50: string (nullable = true)
|-- TWOWD_GT_1: string (nullable = true)
|-- TRACKED_OH: string (nullable = true)
|-- TRACKED\_\_1: string (nullable = true)
|-- OTHER_OHV*: string (nullable = true)
|-- OTHER_OH_1: string (nullable = true)
|-- ATV: string (nullable = true)
|-- ATV_DATESO: string (nullable = true)
|-- MOTORCYCLE: string (nullable = true)
|-- MOTORCYC_1: string (nullable = true)
|-- OTHERWHEEL: string (nullable = true)
|-- OTHERWHE_1: string (nullable = true)
|-- TRACKED**2: string (nullable = true)
|-- TRACKED**3: string (nullable = true)
|-- OTHER_OH_2: string (nullable = true)
|-- OTHER_OH_3: string (nullable = true)
|-- ADMINORG: string (nullable = true)
|-- SECURITYID: string (nullable = true)
|-- DISTRICTNA: string (nullable = true)
|-- FORESTNAME: string (nullable = true)
|-- FIELD_ID: string (nullable = true)
|-- SBS_SYMBOL: string (nullable = true)
|-- ROUTESTATU: string (nullable = true)
|-- GLOBALID: string (nullable = true)
|-- TA_SYMBOL: string (nullable = true)
|-- SHAPE: string (nullable = true)
|-- SHAPELEN: string (nullable = true)
```

Now that we've created a view with our roads data we can use Spatial SQL to query our MVUM data.

```py
sedona.sql("SELECT COUNT(*) AS num FROM roads").show()
```

```md
+------+
| num  |
+------+
|151457|
+------+
```

Since we're going to be hunting for Christmas trees near Santa, Idaho we know we're only interested in roads within the Idaho Panhandle National Forests.

```py
idaho_df = sedona.sql("""
SELECT geometry, NAME as name, FORESTNAME AS forest, SEASONAL AS seasonal, SYMBOL AS symbol FROM roads
WHERE FORESTNAME = 'Idaho Panhandle National Forests'
""")
idaho_df.createOrReplaceTempView("idaho")
idaho_df.show(5)
```

```md
+--------------------+-----------------+--------------------+--------+------+
| geometry           | name            | forest             |seasonal|symbol|
+--------------------+-----------------+--------------------+--------+------+
|LINESTRING (-115....| DOMINION CREEK  |Idaho Panhandle N...|yearlong|     3|
|LINESTRING (-116....| PLACER RIDGE A  |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....| RAINEY HILL     |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-117....| URANIUM         |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....|HAMILTON MOUNTAIN|Idaho Panhandle N...|yearlong|     1|
+--------------------+-----------------+--------------------+--------+------+
```

```py
SedonaKepler.create_map(idaho_df, name="Roads")
```

<Image
  src={'/static/images/finding-tree/kepler_roads.png'}
  alt={'Visualizing roads using SedonaKepler'}
  width={1530}
  height={806}
/>

That's a lot of roads! How are we going to figure out where the best Christmas trees are? Let's look at aerial imagery of the area to find what look like nice tree stands to help narrow down the hunt a bit. We'll use my favorite desktop GIS tool QGIS for this.

## Annotating Aerial Imagery In QGIS

QGIS by default includes tile layers from OpenStreetMap, but we can also add other tile server connections to bring in tile layers from other providers. [This StackExchange answer](https://gis.stackexchange.com/questions/20191/adding-basemaps-in-qgis/217670#217670) describes how to add aerial imagery from Google and Bing to QGIS.

<div style={{ margin: 'auto', maxWidth: '500px' }}>
  <Image
    src={'/static/images/finding-tree/qgis_newtiles.png'}
    alt={'Adding a new tile server to QGIS'}
    width={974}
    height={1178}
  />
</div>

<div style={{ margin: 'auto', maxWidth: '500px' }}>
  <Image
    src={'/static/images/finding-tree/qgis_google.png'}
    alt={'Adding Google aerial imagery'}
    width={1496}
    height={1300}
  />
</div>

Once we add the aerial imagery tile layer we can start to explore the area where
we hope to find some great Christmas trees, in this case in the Idaho Panhandle National
Forests. We're looking for nice heavy green tree stands, not too much vertical variation
for easy access and not near streams and rivers or hiking trails.

Let's add a new layer to our QGIS project so we can keep track of where these tree stands are. I chose a Geopackage layer with Polygon geometry because we're going to draw polygons over the aerial imagery to indicate places that we think will be good for potential trees.

<div style={{ margin: 'auto', maxWidth: '500px' }}>
  <Image
    src={'/static/images/finding-tree/qgis_newlayer.png'}
    alt={'Adding a new layer in QGIS'}
    width={1320}
    height={1652}
  />
</div>

Right-click on the layer and choose "Toggle editing" then click the icon to create
a new polygon. We can now draw polygons over the aerial imagery, each new polygon
will be added as a new feature to our new Geopackage layer. I didn't create any other
fields so the `fid` is a sequence that will be autogenerated. If you wanted to keep
notes about why you thought this area looked good you could do that by adding another
field to the layer.

<Image
  src={'/static/images/finding-tree/qgis_polygon.png'}
  alt={'Adding polygon features'}
  width={1830}
  height={1262}
/>

Once we've annotated a few areas (I found 5 or 6) we can right click on our layer and select "Save as". I exported this layer as GeoJSON, to a file called `idaho_treestands.geojson`.

<div style={{ margin: 'auto', maxWidth: '500px' }}>
  <Image
    src={'/static/images/finding-tree/qgis_exportgeojson.png'}
    alt={'LinkedIn Post'}
    width={1406}
    height={1544}
  />
</div>

## Finding Roads To Our Tree Stands With SedonaDB

Now that we've exported our polygon annotations as GeoJSON it's time to import that data into SedonaDB along with our road data so we can the Forest Service roads that will take us to our tree stands. Return to the "Files" tab in Wherobots Cloud and upload the GeoJSON file we just exported from QGIS.

<Image
  src={'/static/images/finding-tree/files2.png'}
  alt={'The Wherobots Files interface'}
  width={2674}
  height={1096}
/>

To import GeoJSON into SedonaDB we can use Spark's built-in JSON import functionality if we first define the schema used by GeoJSON.

```py
schema = "type string, name string, crs string, features array<struct<type string, geometry string, properties map<string, string>>>"
```

```py
tree_df = sedona.read.option('multiline', True).json(S3_URL_TREES, schema=schema).selectExpr("explode(features) as features").select("features.*").withColumn("geometry", expr("ST_GeomFromGeoJson(geometry)")).withColumn("fid", expr("properties['fid']")).drop("properties").drop("type")
tree_df.createOrReplaceTempView("trees")
tree_df.show(5)
```

```md
+--------------------+---+
| geometry           |fid|
+--------------------+---+
|POLYGON ((-116.52...|  1|
|POLYGON ((-116.54...|  2|
|POLYGON ((-116.12...|  3|
|POLYGON ((-116.16...|  4|
|POLYGON ((-116.09...|  5|
+--------------------+---+
only showing top 5 rows
```

```py
SedonaKepler.create_map(tree_df, name="Tree Stands")
```

<Image
  src={'/static/images/finding-tree/kepler_trees.png'}
  alt={'Visualizing the tree layer with SedonaKepler'}
  width={2008}
  height={822}
/>

As we can see above I found 6 possible tree stands I want to explore. Now we can do a spatial join operation to find the Forest Service roads nearest these tree stands. We could either use the [`ST_Distance` Spatial SQL function](https://docs.wherobots.services/1.2.0/references/sedonadb/vector-data/Function/?h=st_distance#st_distance) to find the closest roads or perhaps use [the `ST_Intersects` predicate](https://docs.wherobots.services/1.2.0/references/sedonadb/vector-data/Predicate/#st_intersects) if we just want to find the roads that go through the tree stands we annotated.

```py
routes1_df = sedona.sql("""
SELECT idaho.geometry, name, forest, seasonal, symbol
FROM idaho, trees
WHERE ST_Intersects(trees.geometry, idaho.geometry)
""")

routes1_df.show()
```

```md
+--------------------+----------------+--------------------+--------+------+
| geometry           | name           | forest             |seasonal|symbol|
+--------------------+----------------+--------------------+--------+------+
|LINESTRING (-116....| PALOUSE DIVIDE |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....| CLARKIA MARBLE |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....|BONNEVILLE POWER|Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....| SWAN PEAK      |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....| CORNWALL CREEK |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....| LACEY CREEK    |Idaho Panhandle N...|yearlong|     1|
|LINESTRING (-116....| CRANBERRY      |Idaho Panhandle N...|yearlong|     1|
+--------------------+----------------+--------------------+--------+------+
```

```py
map = SedonaKepler.create_map(routes1_df, name="routes")
SedonaKepler.add_df(map, tree_df, name="Tree Stands")
```

<Image
  src={'/static/images/finding-tree/kepler_route1.png'}
  alt={'LinkedIn Post'}
  width={2000}
  height={816}
/>

As we can see above most of our tree stands have Forest Service roads leading right though them. We want to expand our route a bit though - we want to know what roads we need to take to turn onto the roads that intersect our tree stands. To do this we can use a SQL subquery to find roads that either intersect or touch the roads that intersect our tree stands.

```py
routes_df = sedona.sql("""
SELECT DISTINCT idaho.geometry, idaho.name, idaho.forest, idaho.seasonal
FROM (
    SELECT idaho.geometry, name, forest, seasonal, symbol
    FROM idaho, trees
    WHERE ST_Intersects(trees.geometry, idaho.geometry)) AS routes, idaho
WHERE ST_Touches(idaho.geometry, routes.geometry)
""")
routes_df.show()
```

```md
+--------------------+------------------+--------------------+--------+
| geometry           | name             | forest             |seasonal|
+--------------------+------------------+--------------------+--------+
|LINESTRING (-116....| CLARKIA MARBLE   |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| CHARLIE CREEK    |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| CRANBERRY        |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| CAMP 3           |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| PALOUSE DIVIDE   |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| 3332 E           |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| HOMESTEAD CREEK  |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| PALOUSE DIVIDE   |Idaho Panhandle N...|seasonal|
|LINESTRING (-116....| EAGLE CREEK      |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| BONNEVILLE POWER |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....|CHARLIE CONNECTION|Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| CRANE            |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| 3332 F           |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| HOODOO           |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| LACEY CREEK      |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| CLARKIA MARBLE   |Idaho Panhandle N...|yearlong|
|LINESTRING (-116....| CORNWALL CREEK   |Idaho Panhandle N...|yearlong|
+--------------------+------------------+--------------------+--------+
```

<Image
    src={'/static/images/finding-tree/kepler_routes.png'}
    alt={'Visualizing routes with SedonaKepler'}
    width={2002}
    height={812}
  />

That gives us a better indication of how to navigate to our tree stands! Next, we'll export our route data to GeoJSON to load back into QGIS to combine with other data from our MVUM dataset to create a navigation map so we can find our Christmas trees!

```py
routes_gdf = geopandas.GeoDataFrame(routes_df.toPandas(), geometry="geometry")
routes_gdf.to_file('routes.geojson', driver='GeoJSON')
````

And here's the final map:

<Image
  src={'/static/images/finding-tree/07-navigation-final.png'}
  alt={'Christmas Tree Hunting Near Santa, Idaho'}
  width={3300}
  height={2550}
/>

## Resources

- [Code on GitHub](https://github.com/johnymontana/30-day-map-challenge/blob/main/notebooks/07-navigation.ipynb)
- [Wherobots Cloud](https://wherobots.com/wherobots-cloud/)


Using aerial imagery and Spatial SQL to explore Forest Service road networks

Finding The Perfect Christmas Tree With USFS Motor Vehicle User Map Data, QGIS, & SedonaDB

My take on the exciting world of spatial database technology plus a look at getting started with Apache Sedona.

Why I Joined Wherobots Plus A Look At Apache Sedona


import Image from 'next/image';
import Subscribe from '../../components/Subscribe';

## Plus Writing Spatial SQL Queries With AWS Athena

In this post we take a look at the recently released public map dataset from the Overture Maps Foundation. We'll see how to select a subset of the data using AWS Athena, how to import the map data into a Neo4j graph database using Neo4j Aura's no-code visual import and data modeling tooling, and finally we'll see how to leverage other no & low-code tooling to create a dashboard using the Neodash graph app and GraphQL API using the Neo4j GraphQL Toolbox.

<Subscribe />

## Overture Maps Foundation Releases Public Dataset

The [Overture Maps Foundation](https://overturemaps.org/) is a collaboration among Meta, Microsoft, TomTom, Amazon, and others with the goal of building a public dataset of map data. It's a Linux Foundation project, similar in structure to the GraphQL Foundation which is a group I'm more familiar with having previously served as Neo4j's representative to the GraphQL Foundation for a few years. 

Overture's goal in their own words is:

> Powering current and next-generation map products by creating reliable, easy-to-use, and interoperable open map data.

<div style={{ margin: 'auto', maxWidth: '500px' }}>
  <Image
    src={'/static/images/overture-graph/overture1.png'}
    alt={'The Overture Maps Foundation'}
    width={518}
    height={428}
  />
</div>

Overture released the first alpha version of their public dataset on July 26th, 2023 so I thought this would be a good time to take a look at the structure of the data in the public dataset as well as see how we can import and analyze the data using Neo4j.

The Overture data consists of four "themes": _places_, _transport_, _admins_, and _buildings_ and is published as Parquet files available via AWS or Azure. Previously I've used Python's pandas library and the Neo4j Python Driver to parse Parquet files into Python dataframes and then import into Neo4j using Cypher (you can see an example of that approach [here where I import the Daylight Earth Table OpenStreetMap distribution into Neo4j](https://github.com/johnymontana/daylight-earth-graph/blob/main/POI_import.ipynb) from Parquet files) but for this project I wanted to stay as code-free as possible and instead use AWS Athena's serverless SQL engine to parse and query the Overture Parquet files then use Neo4j's visual data import tool. This did require that I write a bit of SQL.

You can find more detailed information about each theme in the Overture Maps documentation [here.](https://docs.overturemaps.org/)

## Querying Overture Maps Data With AWS Athena

AWS Athena is a serverless SQL engine that will allow us to execute SQL queries on the Parquet files distributed by Overture.

First, in AWS Athena we need to create a table. I'll start with creating the `places` table, which maps to the Overture places Parquet theme. To create the table in Athena I'll need to specify the schema used for the places theme.

```sql
CREATE EXTERNAL TABLE `places`(
  `id` string,
  `updateTime` string,
  `version` int,
  `names` map<string,array<map<string,string>>>,
  `categories` struct<main:string,alternate:array<string>>,
  `confidence` double,
  `websites` array<string>,
  `socials` array<string>,
  `emails` array<string>,
  `phones` array<string>,
  `brand` struct<names:map<string,array<map<string,string>>>,wikidata:string>,
  `addresses` array<map<string,string>>,
  `sources` array<map<string,string>>,
  `bbox` struct<minX:double,maxX:double,minY:double,maxY:double>,
  `geometry` binary)
PARTITIONED BY (
  `type` varchar(5))
STORED AS PARQUET
LOCATION
  's3://overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=places'
```

And then to trigger building the table.

```sql
MSCK REPAIR TABLE `places`
```

Now that we've created the `places` table, let's write a SQL query to fetch a subset of the Overture places data based on some bouding box. We'll download the results as a CSV file and then use the no-code Neo4j Import tool to create our graph in Neo4j Aura by mapping fields of the CSV file to the property graph model we wish to create using a visual diagramming interface. It's really quite slick!

Let's subset the data based on the bounds of Clark County, Nevada in the US (home of Las Vegas). We can do that using a `bbox` predicate that specifies the lower-left and upper-right coordinates of our bounding box. Some of the attributes in the `places` table are json data so we'll also parse those fields as json and extract the relevant scalar values such as the name, category, and address components using `json_extract` and `json_array_get` SQL functions.

For the places theme each row will contain a point geometry, stored efficiently in the binary WKB (Well Known Binary) format. We'll use the `ST_GeomFromBinary` spatial SQL function to convert that geometry field to WKT (Well Known Text) format and then the `ST_X` and `ST_Y` functions to retrieve the point's longitude and latitude.

```sql
SELECT
    id, updatetime, version, confidence, websites, socials, emails, phones, type,
    ST_X(ST_GeomFromBinary(geometry)) as longitude,
    ST_Y(ST_GeomFromBinary(geometry)) as latitude,
    json_extract_scalar(cast(categories as json), '$.main') as category,
    json_extract(cast(categories as json), '$.alternate') as alt_categories,
    json_extract_scalar(cast(names as json), '$.common[0].value') as name,
    json_extract(json_array_get(cast(sources as json), 0), '$.dataset') as source_dataset,
    json_extract(json_array_get(cast(sources as json), 0), '$.recordid') as source_recordid,
    json_extract(json_array_get(cast(addresses as json), 0), '$.freeform') as street_address,
    json_extract(json_array_get(cast(addresses as json), 0), '$.locality') as locality,
    json_extract(json_array_get(cast(addresses as json), 0), '$.postcode') as postcode,
    json_extract(json_array_get(cast(addresses as json), 0), '$.region') as region,
    json_extract(json_array_get(cast(addresses as json), 0), '$.country') as country
FROM
    places
WHERE
        bbox.minX >  -115.896925
    AND bbox.maxX <  -114.042819
    AND bbox.minY >  35.001857
    AND bbox.maxY <  36.853662
```

<Image
  src={'/static/images/overture-graph/athena3.png'}
  alt={'Querying the Overture Maps places theme with Amazon Athena'}
  width={2954}
  height={1856}
/>

In the AWS Athena console we can select "Download data" to download a CSV file of the results. The file is 26Mb and has about 75k rows.

| "id"                                   | "updatetime"              | "version" | "confidence"          | "websites"                                 | "socials"                                    | "emails" | "phones"         | "type"  | "longitude"    | "latitude"   | "category"       | "alt_categories"                         | "name"                          | "source_dataset" | "source_recordid"      | "street_address"                | "locality"         | "postcode"       | "region" | "country" |
| -------------------------------------- | ------------------------- | --------- | --------------------- | ------------------------------------------ | -------------------------------------------- | -------- | ---------------- | ------- | -------------- | ------------ | ---------------- | ---------------------------------------- | ------------------------------- | ---------------- | ---------------------- | ------------------------------- | ------------------ | ---------------- | -------- | --------- |
| "tmp_9A5E9AA9E18DB0FD734E236DBEEB73B5" | "2023-07-24T00:00:00.000" | "0"       | "0.5052741765975952"  |                                            | "[https://www.facebook.com/140308446009738]" |          |                  | "place" | "-114.3012956" | "35.5715668" | "restaurant"     | "[""landmark_and_historical_building""]" | "Lake Mohave Ranchos Airport"   | """meta"""       | """140308446009738"""  |                                 |                    |                  |          |
| "tmp_7BCFD4C7C12B83CD5355AE15F5A3F262" | "2023-07-24T00:00:00.000" | "0"       | "0.6"                 |                                            |                                              |          |                  | "place" | "-115.0231676" | "36.0496907" |                  | "null"                                   | "Local Vape"                    | """msft"""       | """1970324838862842""" | """7685 Commercial Way Ste I""" | """Henderson"""    | """89011"""      | """NV""" | """US"""  |
| "tmp_FF0104AF0B2048DCD02320B4A096A398" | "2023-07-24T00:00:00.000" | "0"       | "0.26928654313087463" | "[http://ombremonai.com]"                  | "[https://www.facebook.com/108034485157293]" |          | "[+17029101015]" | "place" | "-115.28665"   | "36.15881"   | "beauty_and_spa" | "null"                                   | "Ombré Monai"                   | """meta"""       | """108034485157293"""  | """8751 W Charleston Blvd"""    | """Las Vegas"""    | """89117-5453""" | """NV""" | """US"""  |
| "tmp_D57D0159A70E965E96572CF6D596F1C1" | "2023-07-24T00:00:00.000" | "0"       | "0.8260171413421631"  | "[https://nevadasouthernlivesteamers.com]" | "[https://www.facebook.com/475380855987545]" |          | "[+17023757089]" | "place" | "-114.8539342" | "35.9706829" | "museum"         | "[""history_museum"",""train_station""]" | "Nevada Southern Live Steamers" | """meta"""       | """475380855987545"""  | """601 Yucca St"""              | """Boulder City""" | """89005"""      | """NV""" | """US"""  |

With our CSV file of points of interest downloaded, let's head over to [Neo4j Aura](https://dev.neo4j.com/neo4j-aura) and spin up a free Neo4j instance in the cloud to analyze our place data. After signing in and creating a free tier Neo4j Aura instance we'll use the "Import" tab to create a property graph data model mapped to the places data in our CSV file.

<div style={{ margin: 'auto', maxWidth: '750px' }}>
  <Image
    src={'/static/images/overture-graph/workspace1.png'}
    alt={"Using Neo4j Aura's Workspace developer tooling"}
    width={1386}
    height={606}
  />
</div>

## Data Import Tool

Neo4j Workspace's Import tool is designed around the idea of sketching your property graph data model in an interactive modern diagramming UI and mapping pieces of the data model diagram to fields in your CSV files.

I had worked on [a similar web-based visual graph data import tool prototype](https://github.com/neo4j-contrib/neo4j-csv-import-web) when I first joined Neo4j 8 years ago so it's great to see this functionality available directly in Neo4j Aura.

We can start either by beginning to sketch our property graph data model as we would on a whiteboard, or by selecting the CSV file we want to import. We'll start by selecting the CSV file we downloaded from AWS Athena in the previous step.

<Image
  src={'/static/images/overture-graph/workspace2.png'}
  alt={
    'Using the Neo4j Import Tool to load Overture Maps point of interest data into the Neo4j graph database.'
  }
  width={2524}
  height={1178}
/>

No data is sent to the server, but Import has parsed our CSV file and shows a preview of the values from the first row that we can use to reference when sketching our whiteboard graph model.

<Image
  src={'/static/images/overture-graph/workspace3.png'}
  alt={
    'Sketching a property graph data model using Neo4j Workspace Import tool'
  }
  width={2516}
  height={1176}
/>

If we select the "Add node label" button we can add our first node to the diagram. Let's use this first node label to represent the places of interest themselves so we'll call this node label `PointOfInterest`.

<Image
  src={'/static/images/overture-graph/workspace4.png'}
  slt={
    'Sketching a property graph data model using Neo4j Workspace Import tool'
  }
  width={2516}
  height={1178}
/>

Next, we'll want to specify the properties to include on our `PointOfInterest` nodes from the CSV file. Some of the fields from our CSV file will be extracted out into nodes and relationships of their own (like category, source, and location information) so we don't want to include _all_ fields from the CSV.

We also choose the type of each field and importantly also specify which field defines uniqueness for the node. In other words, which field is the "id" value. In the relational database world this is similar to the concept of "primary key". We need to specify this property to be able to define relationships in our graph.

<Image
  src={'/static/images/overture-graph/workspace5.png'}
  alt={
    'Sketching a property graph data model using Neo4j Workspace Import tool'
  }
  width={2520}
  height={1168}
/>

Adding relationships to the data model is as simple as dragging out from the halo around a node.

<Image
  src={'/static/images/overture-graph/workspace6.png'}
  alt={
    'Sketching a property graph data model using Neo4j Workspace Import tool'
  }
  width={2510}
  height={1170}
/>

In the labeled property graph model used by Neo4j every relationship has both a direction and single "type" that describes the relationship between the two node labels that it connects. We can add the relationship type by clicking on the dashed relationship line.

Since this relationship connects the `PointOfInterest` and `Category` node labels let's define the relationship type as `IN_CATEGORY`. By convention [PascalCase](https://en.wiktionary.org/wiki/Pascal_case) is used when defining node lables and ALL_CAPS_SNAKE_CASE is used for relationship types. You can find more Cypher style guidelines [here](https://neo4j.com/developer/cypher/style-guide/).

<Image
  src={'/static/images/overture-graph/workspace7.png'}
  alt={
    'Sketching a property graph data model using Neo4j Workspace Import tool'
  }
  width={2514}
  height={1178}
/>

To enable the Neo4j Import tool to properly connect nodes we need to specify which fields hold the id values for the "from" and "to" nodes. In this case we have an `id` property for the `PointOfInterest` node which specifies the "from" node and the `category` property value specifies the "to" node.

Notice the green checkmarks that appear as we start to fill out more pieces of the property graph model indicating that we have specified the necessary field mappings.

<Image
  src={'/static/images/overture-graph/workspace8.png'}
  alt={
    'Sketching a property graph data model using Neo4j Workspace Import tool'
  }
  width={2512}
  height={1174}
/>

Next, we continue building up our property graph model, mapping the entities to node labels and the fields that describe them to properties in the graph data model.

<Image
  src={'/static/images/overture-graph/import2.png'}
  alt={
    'Using the Neo4j Import Tool to load Overture Maps point of interest data into the Neo4j graph database.'
  }
  width={3140}
  height={1950}
/>

Here's the final graph model that I ended up with for the Overture dataset.

<Image
  src={'/static/images/overture-graph/import4.png'}
  alt={
    'Using the Neo4j Import Tool to load Overture Maps point of interest data into the Neo4j graph database.'
  }
  width={3122}
  height={1670}
/>

Once we've fully defined our graph data model and mapped to the associated fields of our CSV file we're ready to run the import into our Neo4j Aura instance.

To start the import process we'll click the "Run import" button. Behind the scenes the Neo4j Import tool will generate Cypher queries to create property constraints and then iterate over the data in our CSV file in batches to load data into our Neo4j Aura instance.

<Image
  src={'/static/images/overture-graph/import5.png'}
  alt={
    'Using the Neo4j Import Tool to load Overture Maps point of interest data into the Neo4j graph database.'
  }
  width={1836}
  height={1816}
/>

## What Now?

Now that we've loaded our geospatial data into Neo4j what can we do with it?

We can use the power of Cypher to query for points of interest near us, within certain categories for example. Or more complex queries for location-aware recommendations. For example, let's find all the wineries near me:


```cypher
MATCH (p:PointOfInterest)-[:IN_CATEGORY]->(c:Category {name: "Winery"})
WHERE point.distance(p.location, {latitude: 37.594, longitude: -122.364}) < 100
RETURN p
```

But keeping with our theme of leveraging Neo4j's low-code tools what else do we have at our disposal? I've previously covered Neo4j Bloom which is a great tool for exploring graphs visually, so let's skip Bloom for now and take a look at two of Neo4j's other low-code tools: Neodash and the Neo4j GraphQL Toolbox.

__Building Graph Dashboards With Neodash__

One of the easiest ways to work with data in Neo4j without writing code is to use [NeoDash](https://neodash.graphapp.io/) - a tool from the Neo4j Labs team for building standalone dashboards backed by Neo4j. 

Neodash has support for creating maps and integrating spatial data into dashboards. Here we've just set up a very basic map view so we can explore the Overture data visually.

<Image
  src={'/static/images/overture-graph/neodash1.png'}
  alt={'Creating a dashboard with the Neodash graph app for Neo4j.'}
  width={1174}
  height={553}
/>

__Put The Graph In GraphQL With The Neo4j GraphQL Toolbox__

<Image
  src={'/static/images/overture-graph/graphql1.png'}
  alt={'Neo4j GraphQL Toolbox'}
  width={2421}
  height={1515}
/>

Another low-code tool from Neo4j is the [Neo4j GraphQL Toolbox](https://graphql-toolbox.neo4j.io/) which allows you to create a GraphQL API backed by Neo4j without writing any code. We can just point the Toolbox at our Neo4j Aura instance and generate a fully functional GraphQL API.


This post is just a starting point for working with the Overture maps dataset in Neo4j. It's exciting to see what Overture is doing and I'll definitely be making use of their dataset for future projects. 

## Resources

- [Accessing Overture Maps' Global Dataset using Athena (AWS)](https://feyeandal.me/blog/access_overture_data_using_athena)
- [Athena setup SQL for Overture (all themes)](https://github.com/OvertureMaps/data/blob/main/athena_setup_queries.sql#L77)
- [Extracting data from JSON (AWS Athena Docs)](https://docs.aws.amazon.com/athena/latest/ug/extracting-data-from-JSON.html)
- [Geospatial SQL functions available in Trino](https://trino.io/docs/current/functions/geospatial.html)
- [Bounding Boxes for all US Counties](https://trino.io/docs/current/functions/geospatial.html)
- [Spatial Cypher cheat sheet]()


A look at loading points of interest from the Overture Maps public dataset into the Neo4j graph database, plus writing Spatial SQL queries with AWS Athena.

Importing Overture Maps Data Into Neo4j

A resource for working with geospatial data in Neo4j.

Hi, I'm Will!

Stay Updated

Latest Posts

Deploying Single Instance Neo4j On A VPS With Custom Domain

A Look At Graph Visualization With Sigma React

Finding The Perfect Christmas Tree With USFS Motor Vehicle User Map Data, QGIS, & SedonaDB

Why I Joined Wherobots Plus A Look At Apache Sedona

Importing Overture Maps Data Into Neo4j

Introducing The Spatial Cypher Cheat Sheet

Featured Videos

Featured Projects

Neo4j Agent Memory

Context Graph Demo

Memory Graph Workshop