ny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz
Icon for Socrata external plugin

Query the Data Delivery Network

Query the DDN

The easiest way to query any data on Splitgraph is via the "Data Delivery Network" (DDN). The DDN is a single endpoint that speaks the PostgreSQL wire protocol. Any Splitgraph user can connect to it at data.splitgraph.com:5432 and query any version of over 40,000 datasets that are hosted or proxied by Splitgraph.

For example, you can query the dec_land_acquisition_annual_reports_beginning_1990 table in this repository, by referencing it like:

"ny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz:latest"."dec_land_acquisition_annual_reports_beginning_1990"

or in a full query, like:

SELECT
    ":id", -- Socrata column ID
    "land_acquisition_category", -- Indicates type of acquisition as it relates to the state’s Open Space Conservation Plan and the purpose for which the land was purchased. Open Space Plan Category	Meaning <blank>	Data is Missing AC	Access AFE	Adirondack Forest Easement AFP	Adirondack Forest Preserve AFP/AFE	Adirondack Forest Preserve/ Adirondack Forest Easement ARA	Aquifer Recharge Area BLA	Boundary Line Agreement CFE	Catksill Forest Easement CFP	Catskill Forest Preserve E-ESB	Exceptional Scenic Beauty –  1986 Bond Act EFC	Exceptional Forest Character EFP	Exceptional Forest or Plan  Community E-FWL	Freshwater Wetlands –  1986 Bond Act E-IWA	Inland Waterway Access –  1986 Bond Act E-OS	Open Space - 1986 Bond Act ESB	Exceptional Scenic Beauty ESB-CE	Exceptional Scenic Beauty –  Conservation Easement E-TR	Trailways - 1986 Bond Act E-UC	Unique Area - 1986 Bond Act EX	Exchange FP	Floodplain Preservation FWL	Freshwater Wetlands GW	Greenways HA	Heritage Area IWA	Inland Waterway Access NYC-MOA	NYC Watershed Agreement  (Memorandum of Agreement) OS	Open Space PB	Pine Barrens PFR	Public Fishing Rights PL	Parklands PLA	Enhancement of Public Lands  (access) PLB	Enhancement of Public Lands  (buffer) PLC	Enhancement of Public Lands  (consolidation) Q-AC	Access - 1972 Bond Act Q-FWL	Freshwater Wetlands –  1972 Bond Act Q-SR	Stream Rights - 1972 Bond Act Q-TWL	Tidal Wetlands - 1972 Bond Act SCR	Scenic Resources SLP	Shoreline Protection SP 	 Small Project SRA	State Reforestation Area TOJ	Transfer of Jurisdiction TR	Trailways TWA	Tidal Waterway Access TWL	Tidal Wetlands UA 	Unique Area UC	Unique Character WA	Waterway Access WH	Wildlife Habitat WL	Working Landscape WSP	Watershed Protection  
    "dec_region", -- Administrative region for the New York State Department of Environmental Conservation http://www.dec.ny.gov/about/50230.html
    "county", -- County in which majority of project acreage is located.
    "project_number", -- The DEC assigned number associated with the project.
    "project_name_alias", -- Name given to project during acquisition process. The property name may change after acquisition because the project name often includes the seller’s family name. The project name does not generally match the name eventually given to the land unit after the acquisition is complete.
    "acres_fee", -- Acres of land purchased by DEC Fee simple - an absolute interest in land over which the holder has complete freedom of disposition during his/her life. Note: negative values in Acres (Fee) represent transfers to other state agencies, conveyances to New York City Department of Environmental Protection for drinking water protection (conservation easement retained by DEC), or an exchange of land (a net gain in acreage for DEC). 
    "acres_conservation_easement", -- Acres of land purchase through a conservation easement. Conservation easement - a power invested in a qualified private land conservation organization (often called a "land trust") or government (municipal, county, state or federal) to constrain, as to a specified land area, the exercise of rights otherwise held by a landowner so as to achieve certain conservation purposes. It is an interest in real property established by agreement between a landowner and land trust or unit of government. The conservation easement "runs with the land," meaning it is applicable to both present and future owners of the land. As with other real property interests, the grant of conservation easement is recorded in the local land records; the grant becomes a part of the chain of title for the property.  In New York State, conservation easements are governed by Title 3 of Article 49 of the state’s Environmental Conservation Law, which defines a conservation easement as an easement, covenant, restriction or other interest in real property, created under and subject to the provisions of Title 3 of article 49 of the Environmental Conservation Law which limits or restricts development, management or use of such real property for the purpose of preserving or maintaining the scenic, open, historic, archaeological, architectural, or natural condition, character, significance or amenities of the real property in a manner consistent with the public policy and purpose set forth in section 49-0301 of the Environmental Conservation Law, provided that no such easement shall be acquired or held by the state which is subject to the provisions of Article XIV of the State Constitution. 
    "equivalent_stream_miles", -- The unit of measurement for Public Fishing Rights (PFR) easements, which allow the public to walk along the stream banks for the sole purpose of fishing. Easements along stream banks are normally 33 feet in width, but can vary depending on the purchased rights. One equivalent stream mile is one mile of stream bank on both sides of the stream. If a PFR easement covers one mile on one bank it is considered one half mile of equivalent stream miles. For more information on PFR: http://www.dec.ny.gov/outdoor/7746.html. Equivalent Stream Miles is not used as a unit of measurement for fee acquisition nor other forms of easements (i.e. conservation easement).
    "deed_vesting_date", -- Date the deed was recorded by the county clerk.
    "notes" -- Additional information
FROM
    "ny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz:latest"."dec_land_acquisition_annual_reports_beginning_1990"
LIMIT 100;

Connecting to the DDN is easy. All you need is an existing SQL client that can connect to Postgres. As long as you have a SQL client ready, you'll be able to query ny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz with SQL in under 60 seconds.

This repository is an "external" repository. That means it's hosted elsewhere, in this case at data.ny.gov. When you queryny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz:latest on the DDN, we "mount" the repository using the socrata mount handler. The mount handler proxies your SQL query to the upstream data source, translating it from SQL to the relevant language (in this case SoQL).

We also cache query responses on the DDN, but we run the DDN on multiple nodes so a CACHE_HIT is only guaranteed for subsequent queries that land on the same node.

Query Your Local Engine

Install Splitgraph Locally
bash -c "$(curl -sL https://github.com/splitgraph/splitgraph/releases/latest/download/install.sh)"
 

Read the installation docs.

Splitgraph Cloud is built around Splitgraph Core (GitHub), which includes a local Splitgraph Engine packaged as a Docker image. Splitgraph Cloud is basically a scaled-up version of that local Engine. When you query the Data Delivery Network or the REST API, we mount the relevant datasets in an Engine on our servers and execute your query on it.

It's possible to run this engine locally. You'll need a Mac, Windows or Linux system to install sgr, and a Docker installation to run the engine. You don't need to know how to actually use Docker; sgrcan manage the image, container and volume for you.

There are a few ways to ingest data into the local engine.

For external repositories (like this repository), the Splitgraph Engine can "mount" upstream data sources by using sgr mount. This feature is built around Postgres Foreign Data Wrappers (FDW). You can write custom "mount handlers" for any upstream data source. For an example, we blogged about making a custom mount handler for HackerNews stories.

For hosted datasets, where the author has pushed Splitgraph Images to the repository, you can "clone" and/or "checkout" the data using sgr cloneand sgr checkout.

Mounting Data

This repository is an external repository. It's not hosted by Splitgraph. It is hosted by data.ny.gov, and Splitgraph indexes it. This means it is not an actual Splitgraph image, so you cannot use sgr clone to get the data. Instead, you can use the socrata adapter with the sgr mount command. Then, if you want, you can import the data and turn it into a Splitgraph image that others can clone.

First, install Splitgraph if you haven't already.

Mount the table with sgr mount

sgr mount socrata \
  "ny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz" \
  --handler-options '{
    "domain": "data.ny.gov",
    "tables": {
        "dec_land_acquisition_annual_reports_beginning_1990": "jwv3-emfz"
    }
}'

That's it! Now you can query the data in the mounted table like any other Postgres table.

Query the data with your existing tools

Once you've loaded the data into your local Splitgraph engine, you can query it with any of your existing tools. As far as they're concerned, ny-gov/dec-land-acquisition-annual-reports-beginning-1990-jwv3-emfz is just another Postgres schema.