Query the Data Delivery Network
Query the DDNThe easiest way to query any data on Splitgraph is via the "Data Delivery Network" (DDN). The DDN is a single endpoint that speaks the PostgreSQL wire protocol. Any Splitgraph user can connect to it at data.splitgraph.com:5432 and query any version of over 40,000 datasets that are hosted or proxied by Splitgraph.
For example, you can query the baseline_study_of_food_for_peace_title_ii table in this repository, by referencing it like:
"datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2:latest"."baseline_study_of_food_for_peace_title_ii"or in a full query, like:
SELECT
":id", -- Socrata column ID
"at_anid", -- ANTHROPOMETRIST ID NUMBER
"f10_2", -- F10 2 Do you do anything to the water to make it safer to drink
"ca", -- Cereals
"e29", -- E29 Have you ever been pregnant
"j505_13", -- J505 13 If yes which method
"ed_level_higher",
"f10_4", -- F10 4 Do you do anything to the water to make it safer to drink
"j506_6", -- J506 6 If you are receiving family planning services where do you go for them
"pvo", -- PVO
"hhs_n", -- Household Hunger Scale moderate or severe hunger
"a14_3", -- A14 3 Total Eligible Farmers
"ed_level_secondary",
"f14", -- F14 Show me where members of your household most often wash their hands
"e16", -- E16 Any other fruit or vegetables
"j110_9", -- J110 9 Please list examples of important maternal child care practices
"e39", -- E39 RESULT
"a04", -- A04 SUBCOUNTY NAME
"e12", -- E12 Any foods that are yellow or orange inside
"f10_9", -- F10 9 Do you do anything to the water to make it safer to drink
"c04", -- C04 Any foods made from wheat maize rice sorghum or millet
"j505_12", -- J505 12 If yes which method
"j106_5", -- J106 During your last pregnancy did you see anyone for antenatal care
"j505_6", -- J505 6 If yes which method
"e15", -- E15 Any ripe mangos papayas melon or passionfruit
"j103", -- J103 Are you currently married of living together with a man
"f07", -- F07 Is water available from this source all year round
"int_date", -- Date of Interview
"c06", -- C06 Any vegetables
"f04", -- F04 What is main source of drinking water
"f05", -- F05 Where is that water source located
"improved_water", -- Households using an improved drinking water source
"f02", -- F02 LINE NUMBER OF RESPONDENT
"j1_count2_male", -- Number of unique male caretakers per hh according to roster B08
"j505_8", -- J505 8 If yes which method
"j506_5", -- J506 5 If you are receiving family planning services where do you go for them
"e30", -- E30 Have you ever given birth
"j505_14", -- J505 14 If yes which method
"j110_15", -- J110 15 Please list examples of important maternal child care practices
"c09", -- C09 Any eggs
"j505_10", -- J505 10 If yes which method
"cj", -- Oil fats
"j505_5", -- J505 5 If yes which method
"c12", -- C12 Any cheese yogurt milk or other milk products
"at_sh", -- ANTHRO START HOUR
"female_adult_max", -- Household has at least one female 18 or older
"c08", -- C08 Any meats organ meats or blood
"cg", -- Fish and seafood
"j506_4", -- J506 4 If you are receiving family planning services where do you go for them
"c20", -- C20 In the past 4 weeks did anyone go a whole day and night without eating because there was no food
"j1_count2_female", -- Number of unique male caretakers per hh according to roster B08
"e37", -- E37 WOMAN S HEIGHT CENTIMETERS
"e14", -- E14 Any dark green leafy vegetables
"f10_7", -- F10 7 Do you do anything to the water to make it safer to drink
"ck", -- Sugar honey
"j505_15", -- J505 15 If yes which method
"at_spy", -- ANTHRO SUPERVISOR YEAR
"f10b", -- F10B What is main source of drinking water during the wet season
"j102", -- J102 LINE NUMBER OF CARETAKER FROM HOUSEHOLD ROSTER
"c2021", -- Went all day and night without eating
"f12c", -- F12C Do the children of this household use a different toilet facility as the adult members
"f11", -- F11 What kind of toilet facility do members of your household usually use during the day
"f11b", -- F11B How many households share that daytime facility
"f12", -- F12 What kind of toilet facility do members of your household usually use during the nighttime
"any_sanitation", -- Access to a sanitation facility of any type
"f15", -- F15 Observe presence of water at the place of handwashing
"f01b", -- F01B VILLAGE NUMBER
"f18m", -- MODULE F FINISH MINUTE
"ed_level_none",
"j506_10", -- J506 10 If you are receiving family planning services where do you go for them
"f11a", -- F11A Does your household share the daytime toilet facility with other households
"female_adult", -- Household has at least one female 18 or older
"j300m", -- MODULE J3 START MINUTE
"e11", -- E11 Any foods made from grains
"j102a", -- J102A CHECK IF CARETAKER IS MALE OR FEMALE
"j303_10", -- J303 10 List five critical moments for handwashing
"f00m", -- MODULE F FINISH HOUR
"cc", -- Vegetables
"j301a", -- J301A HOUSEHOLD NUMBER
"j301b", -- J301B VILLAGE NUMBER
"j302", -- J302 LINE NUMBER OF RESONDENT FROM HH ROSTER
"j303_1", -- J303 1 List five critical moments for handwashing
"a14_4", -- A14 4 Total Children Under 5
"j303_2", -- J303 2 List five critical moments for handwashing
"cl", -- Miscellaneous
"soap_water", -- Cleansing agent and water at handwashing station
"j303_3", -- J303 3 List five critical moments for handwashing
"j505_3", -- J505 3 If yes which method
"f11d", -- F11D What kind of facility do the children use in the daytime
"c1617", -- No food at all in the house
"e18", -- E18 Any meat
"j304_cat", -- Are there areas in community that you were unable to visit due to security that are now accessible
"j110_1", -- J110 1 Please list examples of important maternal child care practices
"j505_7", -- J505 7 If yes which method
"j506_3", -- J506 3 If you are receiving family planning services where do you go for them
"j303_4", -- J303 4 List five critical moments for handwashing
"at_anm", -- ANTHROPOMETRIST MONTH
"j506_9", -- J506 9 If you are receiving family planning services where do you go for them
"f10a", -- F10A Does your household use a different source of drInking water during the wet season
"j303_5", -- J303 5 List five critical moments for handwashing
"e05", -- E05 Are you between the ages of 15 49 yrs
"hhs_n2", -- Household Hunger Scale moderate and severe hunger
"j303_6", -- J303 6 List five critical moments for handwashing
"a13y", -- YEAR OF FINAL VISIT
"j303_8", -- J303 8 List five critical moments for handwashing
"e13", -- E13 Potatoes or any other foods made from roots
"a14_1", -- A14 1 Total Number of Visits
"e24", -- E24 Any sugary foods such as chocolate and cake
"ci", -- Milk and milk products
"j104", -- J104 Who usually makes decision about health care for yourself
"f10_3", -- F10 3 Do you do anything to the water to make it safer to drink
"g00h", -- MODULE G START HOUR
"c05", -- C05 Any foods made from roots or tubers
"c01c", -- C01C LINE NUMBER FROM COLUMN 6
"cf", -- Eggs
"dis", -- A05 DISTRICT
"j105", -- J105 Who usually makes decision about health care for your children
"g27h", -- MODULE G FINISH HOUR
"g27m", -- MODULE G FINISH MINUTE
"j106_6", -- J106 During your last pregnancy did you see anyone for antenatal care
"e02", -- E02 LINE NUMBER OF WOMAN FROM HOUSEHOLD ROSTER
"c07", -- C07 Any fruits
"d00m", -- MODULE D START MINUTE
"f00h", -- MODULE F START HOUR
"c00h", -- MODULE C START HOUR
"c01a", -- C01A HOUSEHOLD NUMBER
"male_adult", -- Household has at least one male 18 or older
"j110_8", -- J110 8 Please list examples of important maternal child care practices
"c01b", -- C01B VILLAGE NUMBER
"c10", -- C10 Any fresh or dried fish or shellfish
"c11", -- C11 Any foods made from beans nuts or seeds
"c13", -- C13 Any foods made with oil fat or butter
"j110_13", -- J110 13 Please list examples of important maternal child care practices
"a20y", -- A20 YEAR OF DATA ENTRY
"j303_7", -- J303 7 List five critical moments for handwashing
"id",
"c14", -- C14 Any sugar or honey
"a13m", -- MONTH OF FINAL VISIT
"c22h", -- MODULE C FINISH HOUR
"a20m", -- A20 MONTH OF DATA ENTRY
"a13in", -- INTERVIEWER NUMBER FOR FINAL VISIT
"a14", -- A14 FINAL OUTCOME OF INTERVIEW
"a18", -- A18 FIELD COORDINATOR CODE
"c22m", -- MODULE C FINISH MINUTE
"a20", -- A20 DATA ENTRY OPERATOR CODE
"b00h", -- MODULE B START HOUR
"b00m", -- MODULE B START MINUTE
"a20d", -- A20 DAY OF DATA ENTRY
"d00h", -- MODULE D START HOUR
"d66h", -- MODULE D FINISH HOUR
"a19", -- A19 OFFICE EDITOR CODE
"b21h", -- MODULE B FINISH HOUR
"child_max", -- Household has at least one child under 18
"b21m", -- MODULE B FINISH MINUTE
"d66m", -- MODULE D FINISH MINUTE
"f01a", -- F01A HOUSEHOLD NUMBER
"f06", -- F06 How long does it take to go there get water and come back
"f08", -- F08 In the last two weeks was water unavailable from this source for a day or longer
"j100h", -- MODULE J1 START HOUR
"j100m", -- MODULE J1 START MINUTE
"j101a", -- J101A HOUSEHOLD NUMBER
"a17", -- A17 TEAM LEADER CODE
"f09", -- F09 Do you do anything to the water to make it safer to drink
"f10_1", -- F10 1 Do you do anything to the water to make it safer to drink
"f10_5", -- F10 5 Do you do anything to the water to make it safer to drink
"f10_6", -- F10 6 Do you do anything to the water to make it safer to drink
"a14_5", -- A14 5 Line Number of Respondant to Household Questionnaire
"j101b", -- J101B VILLAGE NUMBER
"f10_8", -- F10 8 Do you do anything to the water to make it safer to drink
"j107", -- J107 Where did you receive antenatal care for this pregnancy
"j110_4", -- J110 4 Please list examples of important maternal child care practices
"f11c", -- F11C Do the children in this household use a different daytime toilet facilites as the adult members
"f12a", -- F12A Does your household share the nighttime toilet facility with any other households
"f12b", -- F12B How many households share the nighttime facility
"f12d", -- F12D What kind of nighttime facility do the children use
"f16", -- F16 Observe presence of soap detergent or other cleansing agent at the place for handwashing
"j110_7", -- J110 7 Please list examples of important maternal child care practices
"f17", -- F17 Observe presence of daytime toilet facility that household said they used
"f18h", -- MODULE F FINISH HOUR
"j300h", -- MODULE J3 START HOUR
"j303_9", -- J303 9 List five critical moments for handwashing
"a14_2", -- A14 2 Total Women 15 49 yrs
"j304", -- J304 Are there areas in community that you were unable to visit due to security that are now accessible
"e00h", -- MODULE E START HOUR
"j305h", -- MODULE J3 FINISH HOUR
"j305m", -- MODULE J3 FINISH MINUTE
"g00m", -- MODULE G START MINUTE
"c00m", -- MODULE C START MINUTE
"c02", -- C02 Was yesterday an unusual or special day
"c15", -- C15 Any other foods such as condiments coffee or tea
"c16", -- C16 In the past 4 weeks was there ever no food to eat due to lack of resources
"a13d", -- DAY OF FINAL VISIT
"e01b", -- E01B VILLAGE NUMBER
"e03a", -- E03A In what month were you born
"a03", -- A03 PARISH NAME
"c17", -- C17 How often did this happen in the past 4 weeks
"e03b", -- E03B In what year were you born
"e04", -- E04 How old are you
"e06", -- E06 CHECK AGE IS BETWEEN 15 49 YRS
"c18", -- C18 In the past 4 weeks did anyone go to sleep hungry because there was not enough food
"c19", -- C19 How often did this happen in the last 4 weeks
"c21", -- C21 How often did this happen in the last 4 weeks
"e19", -- E19 Any eggs
"e20", -- E20 Any fish or seafood
"j106_1", -- J106 During your last pregnancy did you see anyone for antenatal care
"e21", -- E21 Any foods made from beans or seeds
"e22", -- E22 Any cheese yogurt or other milk products
"e23", -- E23 Any foods made with oils fats or butter
"e26", -- E26 Any grubs snails or insects
"e27", -- E27 Any foods made with red palm oil red palm nut or red palm nut pulp sauce
"e28", -- E28 Are you currently pregnant
"j106_2", -- J106 During your last pregnancy did you see anyone for antenatal care
"e31d", -- E31D When was the last time you gave birth DAY
"e31y", -- E31Y When was the last time you gave birth YEAR
"e33h", -- MODULE E FINISH HOUR
"e33m", -- MODULE E FINISH MINUTE
"vn", -- A02 VILLAGE NUMBER
"e34", -- E34 ANTHROPOMETRY LINE NUMBER OF WOMAN FROM ROSTER
"e36", -- E36 WOMAN S AGE IN YEARS
"j106_3", -- J106 During your last pregnancy did you see anyone for antenatal care
"e38", -- E38 WOMAN S WEIGHT KILOGRAMS
"j500h", -- MODULE J5 START HOUR
"j500m", -- MODULE J5 START MINUTE
"j501b", -- J501B VILLAGE NUMBER
"j502", -- J502 LINE NUMBER OF WOMAN FROM ROSTER
"j503", -- J503 Do you know of a place to go to for family planning
"j504", -- J504 Are you and your partner doing something to delay or prevent you from getting pregnant
"j505_1", -- J505 1 If yes which method
"j505_11", -- J505 11 If yes which method
"j106_4", -- J106 During your last pregnancy did you see anyone for antenatal care
"j506_1", -- J506 1 If you are receiving family planning services where do you go for them
"j507", -- J507 If not what is the reason
"j508h", -- MODULE J5 FINISH HOUR
"at_id", -- ANTHRO QUESTIONNAIRE NUMBER
"j108", -- J108 How many times did you receive antenatal care during this pregnancy
"at_vn", -- ANTHRO VILLAGE NUMBER
"at_sm", -- ANTHRO START MINUTE
"at_any", -- ANTHROPOMETRIST YEAR
"at_spd", -- ANTHRO SUPERVISOR DAY
"at_spm", -- ANTHRO SUPERVISOR MONTH
"at_fh", -- ANTHRO FINISH HOUR
"at_fm", -- ANTHRO FINISH MONTH
"member_id", -- Member ID for module J J102
"birth_date", -- Woman s date of birth
"eligible_women", -- Women eligible for Module E between 15 and 49 years of age
"j109", -- J109 Did your partner accompany you during your antenatal visits
"education_level",
"strata", -- Strata
"hh", -- A01 HOUSEHOLD NUMBER
"cluster", -- Cluster
"j110_2", -- J110 2 Please list examples of important maternal child care practices
"j110_3", -- J110 3 Please list examples of important maternal child care practices
"hhwt", -- Household Weight
"wmwt", -- Sampling weight for women 15 49 adjusted for module E non response
"j110_5", -- J110 5 Please list examples of important maternal child care practices
"wmwt_j5", -- Sampling weight for women 15 49 adjusted for module J5 non response
"j1wt", -- Sampling weight for primary caretakers adjusted for module J1 non response
"j110_6", -- J110 6 Please list examples of important maternal child care practices
"j110_10", -- J110 10 Please list examples of important maternal child care practices
"cd", -- Fruits
"critical_handwashing_cat", -- Knows 3 of 5 critical moments for handwashing
"j110_11", -- J110 11 Please list examples of important maternal child care practices
"j110_12", -- J110 12 Please list examples of important maternal child care practices
"ch", -- Pulses legumes nuts
"hdds", -- Average Household Dietary Diversity Score HDDS
"hhs", -- Household Hunger Scale
"improved_sanitation", -- Households using improved sanitation facilities
"j110_14", -- J110 14 Please list examples of important maternal child care practices
"j111h", -- MODULE J1 FINISH HOUR
"j111m", -- MODULE J2 FINISH MINUTE
"critical_handwashing", -- Number kown of critical moments for handwashing out of 5
"e00m", -- MODULE E START MINUTE
"e01a", -- E01A HOUSEHOLD NUMBER
"c1819", -- Went to bed hungry
"e17", -- E17 Any liver kidney heart or other organ meats blood
"e25", -- E25 Any condiments such as chilies spices herbs or fish powder
"e31m", -- E31M When was the last time you gave birth MONTH
"j501a", -- J501A HOUSEHOLD NUMBER
"j505_2", -- J505 2 If yes which method
"j505_4", -- J505 4 If yes which method
"j505_9", -- J505 9 If yes which method
"j506_2", -- J506 2 If you are receiving family planning services where do you go for them
"j506_7", -- J506 7 If you are receiving family planning services where do you go for them
"j506_8", -- J506 8 If you are receiving family planning services where do you go for them
"j508m", -- MODULE J5 FINISH MINUTE
"child", -- Household has at least one child under 18
"ed_level_preprimary",
"at_hh", -- ANTHRO HOUSEHOLD NUMBER
"ed_level_primary",
"at_and", -- ANTHROPOMETRIST DAY
"ce", -- Meats organs blood
"gendered_hh", -- Gendered Household type
"male_adult_max", -- Household has at least one male 18 or older
"wom_age", -- Women s age in completed years
"cb", -- Root and tubers
"at_spid" -- ANTHRO SUPERVISOR ID NUMBER
FROM
"datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2:latest"."baseline_study_of_food_for_peace_title_ii"
LIMIT 100;Connecting to the DDN is easy. All you need is an existing SQL client that can connect to Postgres. As long as you have a SQL client ready, you'll be able to query datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2 with SQL in under 60 seconds.
Query Your Local Engine
Install Splitgraph Locally
bash -c "$(curl -sL https://github.com/splitgraph/splitgraph/releases/latest/download/install.sh)"Read the installation docs.
Splitgraph Cloud is built around Splitgraph Core (GitHub), which includes a local Splitgraph Engine packaged as a Docker image. Splitgraph Cloud is basically a scaled-up version of that local Engine. When you query the Data Delivery Network or the REST API, we mount the relevant datasets in an Engine on our servers and execute your query on it.
It's possible to run this engine locally. You'll need a Mac, Windows or Linux system to install sgr, and a Docker installation to run the engine. You don't need to know how to actually use Docker; sgrcan manage the image, container and volume for you.
There are a few ways to ingest data into the local engine.
For external repositories, the Splitgraph Engine can "mount" upstream data sources by using sgr mount. This feature is built around Postgres Foreign Data Wrappers (FDW). You can write custom "mount handlers" for any upstream data source. For an example, we blogged about making a custom mount handler for HackerNews stories.
For hosted datasets (like this repository), where the author has pushed Splitgraph Images to the repository, you can "clone" and/or "checkout" the data using sgr cloneand sgr checkout.
Cloning Data
Because datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2:latest is a Splitgraph Image, you can clone the data from Spltgraph Cloud to your local engine, where you can query it like any other Postgres database, using any of your existing tools.
First, install Splitgraph if you haven't already.
Clone the metadata with sgr clone
This will be quick, and does not download the actual data.
sgr clone datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2Checkout the data
Once you've cloned the data, you need to "checkout" the tag that you want. For example, to checkout the latest tag:
sgr checkout datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2:latestThis will download all the objects for the latest tag of datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2 and load them into the Splitgraph Engine. Depending on your connection speed and the size of the data, you will need to wait for the checkout to complete. Once it's complete, you will be able to query the data like you would any other Postgres database.
Alternatively, use "layered checkout" to avoid downloading all the data
The data in datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2:latest is 0 bytes. If this is too big to download all at once, or perhaps you only need to query a subset of it, you can use a layered checkout.:
sgr checkout --layered datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2:latestThis will not download all the data, but it will create a schema comprised of foreign tables, that you can query as you would any other data. Splitgraph will lazily download the required objects as you query the data. In some cases, this might be faster or more efficient than a regular checkout.
Read the layered querying documentation to learn about when and why you might want to use layered queries.
Query the data with your existing tools
Once you've loaded the data into your local Splitgraph Engine, you can query it with any of your existing tools. As far as they're concerned, datahub-usaid-gov/baseline-study-of-food-for-peace-title-ii-d7ka-5mp2 is just another Postgres schema.