Input Processing
Each model can override parts of Aphrodite's input processing pipeline via INPUT_REGISTRY
and MULTIMODAL_REGISTRY
.
Currently, this mechanism is only utilized in multi-modal models for preprocessing multi-modal input data in addition to input prompt, but it can be extended to text-only language models when needed.
Input Processing Pipeline
- Input data is passed to
AphroditeEngine
(orAsyncAphrodite
for asynchronous inference). - Tokenize the data if necessary.
- Process the inputs using
INPUT_REGISTRY.process_input
. - Send the processed inputs to
ExecutorBase
. - Distribute the inputs via
WorkerBase
toModelRunnerBase
. - If the data contains multi-modal data, convert it into keyword arguments using
MULTIMODAL_REGISTRY.map_input
.
- For example, convert a
PIL.Image.Image
input to its pixel values for a vision model.