mirror of
https://github.com/NetherlandsForensicInstitute/hansken-extraction-plugin-sdk-documentation.git
synced 2026-05-16 07:06:51 +00:00
101 lines
4.4 KiB
Plaintext
101 lines
4.4 KiB
Plaintext
# Using Transformers for on-demand execution
|
|
|
|
## What are Transformers?
|
|
|
|
Transformers are methods inside a plugin that can be called remotely at any moment.
|
|
This allows for live plugin execution independent of extraction.
|
|
Examples on how transformers could be used:
|
|
- For searching images using text (i.e. a purple car).
|
|
- For translating text in traces so that an investigator can read the text in their preferred language in a UI.
|
|
- For converting speech to text.
|
|
|
|
## How do Transformers work in Hansken?
|
|
|
|
Transformers can be implemented in extraction plugins. Using the Hansken REST API, calls can be made to a
|
|
specific plugin's transformer by specifying the transformer one wishes to call as well as its arguments.
|
|
Hansken can automatically discover transformers before plugins are actually started.
|
|
Once it has received a request for invoking a transformer it can choose to start the plugin's Docker container if it
|
|
is not already started and send the request once it is up and running.
|
|
|
|
## Developing Transformers
|
|
|
|
This section assumes you use the same setup as is used in the [Extraction Plugin Examples](https://github.com/NetherlandsForensicInstitute/hansken-extraction-plugin-template-python/).
|
|
|
|
A transformer can be easily defined by using the transformer decorator. By creating a method inside your plugin and
|
|
decorating it with the transformer decorator it will automatically be made available to be remotely called.
|
|
|
|
```python
|
|
from hansken_extraction_plugin.api.extraction_plugin import ExtractionPlugin
|
|
from hansken_extraction_plugin.decorators.transformer import transformer
|
|
class Plugin(ExtractionPlugin):
|
|
|
|
def plugin_info(self):
|
|
...
|
|
|
|
def process(self, trace, data_context):
|
|
...
|
|
|
|
@transformer
|
|
def translate_text(self, text: str, language: str) -> str:
|
|
# To implement: Translate the text here.
|
|
return "Translated text"
|
|
|
|
```
|
|
|
|
## Limitations on Transformers
|
|
|
|
However, there are some limitations on which methods can be turned into a decorator method:
|
|
|
|
- Transformers may only be defined on methods of a class that derives (indirectly) from BaseExtractionPlugin.
|
|
- Note: ExtractionPlugin derives from BaseExtractionPlugin and is therefore allowed.
|
|
- Transformer may not be static methods.
|
|
- All parameters and the return type must be annotated with type hints (except the `self` parameter).
|
|
- Parameters may not be positional-only or contain variable parameters like `*args` or `**kwarg`.
|
|
- Parameters and return types may only be of the following (returning None is not supported):
|
|
- bool
|
|
- int
|
|
- float
|
|
- str
|
|
- bytes
|
|
- bytearray
|
|
- datetime.datetime (The datetime will be converted to unix time and then converted to datetime with the zone "UTC". So the zone id is always("UTC"))
|
|
- hansken.util.GeographicLocation
|
|
- hansken.util.Vector
|
|
- typing.Sequence
|
|
- typing.Mapping
|
|
|
|
Upon starting a plugin every method decorated with the transformer decorator will be automatically validated to see if they adhere to these requirements. An exception will be thrown when a method does not adhere to all of these requirements.
|
|
|
|
## Error handling in Transformers
|
|
|
|
If at some point a transformer wishes to signal to the caller of the transformer that something has gone wrong
|
|
it can simply throw a suitable exception. Any exceptions being thrown will automatically be propagated to the client
|
|
calling the transformer by wrapping the exception in a gRPC exception. The stack trace will be provided as well for
|
|
debugging purposes.
|
|
|
|
## Testing transformers
|
|
|
|
For more information on testing transformers, see the 'Testing Transformers' section on the [page on testing plugins](testing.md).
|
|
|
|
## Calling transformer with /tools/transformers
|
|
|
|
This REST Call calls a specified transformer method on a given tool/plugin with supplied arguments.
|
|
This endpoint allows a transformer to be executed on demand, outside an extraction. The transformer call can be used to transform user input to an output. A tool/plugin can have multiple transformer methods.
|
|
To use /tools/transformers REST CALL you need to know the following (see extractions tool or use the rest call/tools to get the information below):
|
|
- The name of the plugin/tool
|
|
- The transformer name
|
|
- The names of the parameters (arguments)
|
|
|
|
Example JSON request body for the transformer above:
|
|
|
|
```JSON
|
|
{
|
|
"arguments": {
|
|
"text": "value1",
|
|
"language": "value2"
|
|
},
|
|
"tool": "ExampleTool",
|
|
"transformer": "translate_text"
|
|
}
|
|
```
|