Microservices - Serving

Shakudo services are long-running jobs with an exposed endpoint. Services are only available from within the cluster by default, but can also be reached externally using a jwt token (explained below). Services are the best way to make an application available or to enable real-time data processing or streaming appplications on Shakudo.

Creating a Microservice

Services can be created from the Shakudo landing page by clicking the "start a service" button.

Alternatively, services can be created from the service page by clicking "create service".

Basic settings

A basic service can be started by simply providing the service name (automatically generated by default), the desired service endpoint, an environment configuration (which work the same way as in Shakudo Sessions), and a path to the service configuration yaml file, relative to the git repository associated with the service (configurable in the advanced tab)

Advanced settings

The advanced tab allows configuring some more advanced service parameters, described in the table below:

Name	Description
Port	Enter a port to expose between 1 and 65535. The default for Shakudo Platform services is port 8787.
Min Replicas	Minimum amount of replicas running the service simultaneously
Max Replicas	Maximum amount of replicas running the service simultaneously
Git repository	The git repository associated with the service
Branch	Git branch name for the service
Commit	Git commit ID hash to use for the service

The git repository, branch and commit will be used to clone a project into the service environment, and the serivce yaml will be located and run from the root of the cloned project.

Additional arbitrary parameters for the service runner configuration can be set in the Parameters tab.

Parameters

Similarly to Jobs, you can add Parameters to inject some additional information, which you can use in your code. When you attach Parameters, they will be available as environment variables, in upper case, with all non-alphanumeric characters replaced by _. For example, if you add an parameter with name "model-name", it will be accessible as the MODEL_NAME in the Service's environment variables.

To maintain backward compatibility with legacy services, Parameters are also available with the HYPERPLANE_JOB_PARAMETER_. Using the example above, this would be HYPERPLANE_JOB_PARAMETER_MODEL_NAME.

Note that these values are stored in plain text, it's recommended that you use Secrets for API keys, keyfile strings, access credentials, and other secret values.

Secrets

When you attach Secrets, they will be available both as environment variables and as a file.

In the case of environment variable, the secret key will be converted to upper case, with all non-alphanumeric characters replaced by _, and prefixed with HYPERPLANE_CUSTOM_SECRET_KEY_. For example, if you add an parameter with name "openai-key", it will be accessible as the HYPERPLANE_CUSTOM_SECRET_KEY_OPENAI_KEY in the Service's environment variables.

Secrets are available as files, in the format of /etc/hyperplane/secrets/{secret_name}/{secret_key}.

Starting with a bash script

If you only need to run a single step consisting of a bash script to start your app, select "Shell" instead of "Multi-step" in the Job creation dialogue.

As you would for multi-step pipeline YAMLs, specify the path to the bash script relative to the root of your git repo.

An example bash script might look like the following:

#!/bin/bash
set -e
PROJECT_DIR="$(cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

cd "$PROJECT_DIR"

pip install foo-package==bar-version
python ./app.py

Note the shebang for bash. The -e flag detects errors in the currently running script, and stops the script when one of the commands within returns a non-zero status. PROJECT_DIR finds the current directory of the current bash script. It is useful when you want to reference things relative to the current bash script.

This bash script will start your microservice (e.g. a Flask app) and run until app.py exits with an error. In the case of an error, the pod will be restarted and the script will rerun from the top.

Service actions

The service actions menu can be used to operate on a current or past service.

Cancel

A current service can be cancelled by choosing the "cancel" action in the action menu.

Clone

A new service can be created based on the configuration of an existing service by choosing the "clone" action. This can also be used on past services that have been cancelled to recreate a service of the same type, or to use the service settings as a template for faster iteration.

Restart

Restarting a service (which will cause it to pull from the associated repository as per its settings) can be done by selecting the "restart" action.

Updating a service

Shakudo does not yet support a single-action update operation on services. Instead, when a service configuration is modified, services are updated by first closing a service and then starting it anew with an updated configuration. For updates to the service code (from the repository associated with the repository), the service should be restarted with the restart action.

Accessing a service from outside the cluster

Services can be accessed externally using a jwt token using the bearer method.

• Prepare to make a POST request to https://{your_shakudo_domain}/auth/realms/{realm_name}/protocol/openid-connect/token, for example using curl or postman
  • {your_shakudo_domain} is the domain at which your keycloak is available, which is usually the same domain at which you access your cluster running Shakudo
  • You must know your {realm_name} to obtain the jwt token.
• Set the Content-Type header to application/x-www-form-urlencoded
• Set the following parameters in the request body:
  • client_id: the client used to get the token (probably istio)
  • grant_type: value should be password
  • username: Your username
  • password: Your password

note

The Access Type must be public to obtain a jwt this way

The response jwt can then be used to access service endpoints: simply add Authorization: Bearer {token} to your Headers, with {token} being the jwt obtained in the previous step.