KFServing

KFServing enables serverless inferencing on Kubernetes and provides performant, high abstraction interfaces for common machine learning (ML) frameworks like TensorFlow, XGBoost, scikit-learn, PyTorch, and ONNX to solve production model serving use cases.

You can use KFServing to do the following:

• Provide a Kubernetes Custom Resource Definition for serving ML models on arbitrary frameworks.

• Enable a simple, pluggable, and complete story for your production ML inference server by providing prediction, pre-processing, post-processing and explainability out of the box.

Our strong community contributions help KFServing to grow. We have a Technical Steering Committee driven by Google, IBM, Microsoft, Seldon, and Bloomberg. to give us feedback!

KFServing works with Kubeflow 0.7. Kustomize installation files are located in the manifests repo.

Examples

• TensorFlow
• scikit-learn
• Custom
• GPU
• Pipelines
• Azure
• S3

• Transformer (pre/post processing)

We frequently add examples to our GitHub repo.

Learn more

• Join our working group for meeting invitations and discussion.
• .
• API docs.
• .
• KFServing 101 slides.

Knative Serving (v0.8.0 +) and Istio (v1.1.7+) should be available on your Kubernetes cluster.

Read more about .

KFServing installation using kubectl

• Install the SDK.

pip install kfserving

• to use the KFServing SDK to create, patch, roll out, and delete a KFServing instance.

Contribute

• .