Custom Splitfile commands
It is possible to define custom Splitfile commands that follow the normal Splitfile execution semantics (cache invalidation and idempotency), allowing users to write their own transformation or data import stages.
A custom command has to extend the
and implement at least one method,
execute(), that runs the command and
returns the command context hash which is combined with the hash of the
previous image to create the hash of the new image.
Additionally, the custom command can also implement
calc_hash() if it's able to calculate the
command context hash without actually running the command.
The Python class can then be registered in the
.sgconfig file as follows:
For example, for a simple data importer that loads a file from HTTP,
calc_hash() could return the hash of the file's last modified
execute() would perform the actual computation-heavy
Custom commands are currently not supported by provenance tracking.
See the Python API reference for more information on building custom Splitfile commands.