Depending on the plugin, Splitgraph can perform data integration in one or more of the three modes:
Load the data from a source into a Splitgraph repository. This creates a new version ("tag") of the data with the source dataset reloaded from scratch.
All data sources support a full reload.
Incremental load (change data capture)
Some data sources also support incremental replication: only copying over new or changed data since the last time the ingestion ran. In some cases, you might need to specify a monotonically increasing column to be used as a "replication cursor".
Query the data live (at source) without ingestion. This is also sometimes called
"data federation". Splitgraph will create a special tag in a repository called
:live that you can reference in order to run the query against the original
SELECT * FROM "some/repo:live".some_table