Hazardous Waste Portal Manifest Metadata

Note: Please use the following view to be able to see the entire Dataset Description: https://data.ct.gov/Environment-and-Natural-Resources/Hazardous-Waste-Portal-Manifest-Metadata/x2z6-swxe

Dataset Description Outline (5 sections)

• INTRODUCTION

• WHY USE THE CONNECTICUT OPEN DATA PORTAL MANIFEST METADATA DATASET INSTEAD OF THE DEEP DOCUMENT ONLINE SEARCH PORTAL ITSELF?

• WHAT MANIFESTS ARE INCLUDED IN DEEP’S MANIFEST PERMANENT RECORDS ARE ALSO AVAILABLE VIA THE DEEP DOCUMENT SEARCH PORTAL AND CT OPEN DATA?

• HOW DOES THE PORTAL MANIFEST METADATA DATASET RELATE TO THE OTHER TWO MANIFEST DATASETS PUBLISHED IN CT OPEN DATA?

• IMPORTANT NOTES

INTRODUCTION

• All of DEEP’s paper hazardous waste manifest records were recently scanned and “indexed”. • Indexing consisted of 6 basic pieces of information or “metadata” taken from each manifest about the Generator and stored with the scanned image. The metadata enables searches by: Site Town, Site Address, Generator Name, Generator ID Number, Manifest ID Number and Date of Shipment.

• All of the metadata and scanned images are available electronically via DEEP’s Document Online Search Portal at: https://filings.deep.ct.gov/DEEPDocumentSearchPortal/

• Therefore, it is no longer necessary to visit the DEEP Records Center in Hartford for manifest records or information.

• This CT Data dataset “Hazardous Waste Portal Manifest Metadata” (or “Portal Manifest Metadata”) was copied from the DEEP Document Online Search Portal, and includes only the metadata – no images.

WHY USE THE CONNECTICUT OPEN DATA PORTAL MANIFEST METADATA DATASET INSTEAD OF THE DEEP DOCUMENT ONLINE SEARCH PORTAL ITSELF?

The Portal Manifest Metadata is a good search tool to use along with the Portal. Searching the Portal Manifest Metadata can provide the following advantages over searching the Portal:

• faster searches, especially for “large searches” - those with a large number of search returns unlimited number of search returns (Portal is limited to 500);

• larger display of search returns;

• search returns can be sorted and filtered online in CT Data; and

• search returns and the entire dataset can be downloaded from CT Data and used offline (e.g. download to Excel format)

• metadata from searches can be copied from CT Data and pasted into the Portal search fields to quickly find single scanned images.

The main advantages of the Portal are:

• it provides access to scanned images of manifest documents (CT Data does not); and

• images can be downloaded one or multiple at a time.

WHAT MANIFESTS ARE INCLUDED IN DEEP’S MANIFEST PERMANENT RECORDS ARE ALSO AVAILABLE VIA THE DEEP DOCUMENT SEARCH PORTAL AND CT OPEN DATA?

All hazardous waste manifest records received and maintained by the DEEP Manifest Program; including:

• manifests originating from a Connecticut Generator or sent to a Connecticut Destination Facility including manifests accompanying an exported shipment

• manifests with RCRA hazardous waste listed on them (such manifests may also have non-RCRA hazardous waste listed)

• manifests from a Generator with a Connecticut Generator ID number (permanent or temporary number)

• manifests with sufficient quantities of RCRA hazardous waste listed for DEEP to consider the Generator to be a Small or Large Quantity Generator

• manifests with PCBs listed on them from 2016 to 6-29-2018.

• Note: manifests sent to a CT Destination Facility were indexed by the Connecticut or Out of State Generator. Searches by CT Designated Facility are not possible unless such facility is the Generator for the purposes of manifesting.

All other manifests were considered “non-hazardous” manifests and not scanned. They were discarded after 2 years in accord with DEEP records retention schedule. Non-hazardous manifests include:

• Manifests with only non-RCRA hazardous waste listed

• Manifests from generators that did not have a permanent or temporary Generator ID number

• Sometimes non-hazardous manifests were considered “Hazar

Querying over HTTP

Splitgraph serves as an HTTP API that lets you run SQL queries directly on this data to power Web applications. For example:

curl https://data.splitgraph.com/sql/query/ddn \
    -H "Content-Type: application/json" \
    -d@-<<EOF
{"sql": "
    SELECT *
    FROM \"ct-gov/hazardous-waste-portal-manifest-metadata-x2z6-swxe\".\"hazardous_waste_portal_manifest_metadata\"
    LIMIT 100 
"}
EOF

See the Splitgraph documentation for more information.