Install with HTTP/SSL with Models and Advanced Options
This configuration combines an SSL configuration with additional embedding models and the configuration file. For a more advanced configuration, you can optionally choose to specify the HTTP port, container version, and or container name.
In this example, your desired ONNX Pipeline models are in /home/opc/models and your config file is /home/opc/config/config.json.
-
Configure and start the container with the models provided in the configuration file. Notice the file locations specified when running the
configSetupscript.Note:
If a container is already running and you try to start a new container with the same configuration, then the old container will automatically be stopped and replaced with the new container.
If you try to start a new container with a different configuration than that of the running container, then the container start will likely fail, and you will need to stop the existing container.
mkdir /home/opc/privateai mkdir /home/opc/secrets mkdir /home/opc/models mkdir /home/opc/config export PRIVATE_DIR=/home/opc/privateai export SECRETS_DIR=/home/opc/secrets ./secretsSetup.sh -s $SECRETS_DIR ./configSetup.sh -d $PRIVATE_DIR -s $SECRETS_DIR -m /home/opc/models -c /home/opc/config/config.json ./containerSetup.sh -d $PRIVATE_DIRYou can optionally specify the port, version, and container name when running the
containerSetupscript by replacing the last line, as in the following:podman stop privateai ./containerSetup.sh -d $PRIVATE_DIR -p 9443 -v 26.2.1.0.0 -n everything4 podman ps -
Define the value of the
API_KEYto make the curl SSL commands easier and define theAPI_KEYvalue:export PORT=9443 export API_KEY=$(cat $SECRETS_DIR/api-key) -
Provide the digital certificate as a parameter.
The same SSL curl commands work for both local and remote client machines.
curl -i --cacert $SECRETS_DIR/cert.pem https://$HOST:$PORT/health curl --header "Authorization: Bearer $API_KEY" --cacert $SECRETS_DIR/cert.pem https://$HOST:$PORT/v1/models curl -X POST --header "Authorization: Bearer $API_KEY" -H "Content-Type: application/json" -d '{"model": "tinybert", "input":["This is a phrase to vectorize"]}' --cacert $SECRETS_DIR/cert.pem https://$HOST:$PORT/v1/embeddings curl --header "Authorization: Bearer $API_KEY" --cacert $SECRETS_DIR/cert.pem https://$HOST:$PORT/metrics/embeddings_call_latency