Custom Splitfile commands
It is possible to define custom Splitfile commands that follow the normal Splitfile execution semantics (cache invalidation and idempotency), allowing users to write their own transformation or data import stages.
A custom command has to extend the PluginCommand
class
and implement at least one method, execute()
, that runs the command and
returns the command context hash which is combined with the hash of the
previous image to create the hash of the new image.
Additionally, the custom command can also implement calc_hash()
if it's able to calculate the
command context hash without actually running the command.
The Python class can then be registered in the .sgconfig
file as follows:
[commands]
COMMAND_NAME=module.with.my.CommandClass
For example, for a simple data importer that loads a file from HTTP,
calc_hash()
could return the hash of the file's last modified
timestamp and execute()
would perform the actual computation-heavy
data import.
Custom commands are currently not supported by provenance tracking.
See the Python API reference for more information on building custom Splitfile commands.