This doc explains how to set up a development environment for Numaflow.
go
1.20+.git
.kubectl
.protoc
3.19 for compiling protocol buffers.pandoc
2.17 for generating API markdown.Node.js®
for running the UI.yarn
.- A local Kubernetes cluster for development usage, pick either one of
k3d
,kind
, orminikube
.
# Install kind on macOS
brew install kind
# Create a cluster with default name kind
kind create cluster
# Get kubeconfig for the cluster
kind export kubeconfig
Please install the metrics server if your local Kubernetes cluster does not bring it by default (e.g., Kind). Without the metrics-server, we will not be able to see the pods in the UI.
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl patch -n kube-system deployment metrics-server --type=json -p '[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--kubelet-insecure-tls"}]'
-
make start
Build the source code, image, and install the Numaflow controller in thenumaflow-system
namespace. -
make build
Binaries are placed in./dist
. -
make manifests
Regenerate all the manifests after making any base manifest changes. This is also covered bymake codegen
. -
make codegen
Run after making changes to./pkg/api/
. -
make test
Run unit tests. -
make test-*
Run one e2e test suite. e.g.make test-kafka-e2e
to run the kafka e2e suite. -
make Test*
Run one e2e test case. e.g.make TestKafkaSourceSink
to run theTestKafkaSourceSink
case in the kafka e2e suite. -
make image
Build container image, and import it tok3d
,kind
, orminikube
cluster if correspondingKUBECONFIG
is sourced. -
make docs
Convert the docs to GitHub pages, check if there's any error. -
make docs-serve
Start an HTTP server on your local to host the docs generated Github pages.