Zarf massively simplifies the setup & administration of kubernetes clusters, cyber systems & workloads that support DevSecOps "across the air gap".

It provides a static go binary (can run anywhere) CLI that can pull, package, and install all the things your clusters need to run along with any necessary resources to standup infrastructure such as Terraform. Zarf also caches downloads (for speed), hashes packages (for security), and can even install the Kubernetes cluster itself if you want it to.

Zarf runs on a bunch of operating systems and aims to support configurations ranging from "I want to run one, simple app" to "I need to support & dependency control a bunch of internet-disconnected clusters".

Zarf was theorized and initially demonstrated in Naval Postgraduate School reasearch:

A DEVSECOPS APPROACH FOR DEVELOPING AND DEPLOYING CONTAINERIZED CLOUD-BASED SOFTWARE ON SUBMARINES.

Its origin story remains to be told but is of the likes seen in Kessel Run.

Most of the software ecosystem assumes your systems have access to the internet. The world (for good reasons) has become more and more dependent upon Software as a Service (SaaS), which assumes a robust connection to the internet and a willingness to inherently trust 3rd party providers. Although this makes sense for most of the world, there are certain SECURE systems that must operate either fully disconnected, semi-disconnected, or might need the ability to disconnect in case of emergencies (like while under an active cyber attack). Although only a small percentage of systems, these SECURE systems make up some of the most vital systems in the world, such as Aerospace and Defense, Finance, Healthcare, Energy, Water, Sewage, and many Federal, Local, and State Government systems.

These SECURE systems need a way to continuously and securely deliver software too. Zarf exists to make it easy for open-source, commercial, and organic software solutions to be delivered to secure and disconnected systems. Although such systems might be small in number, they represent many of the most important systems in the world.

Zarf allows you to bundle portions of "the internet" into a single package to be installed later following specific instructions. A Zarf package is really just a single file that includes everything you would need to manage a system or capability while fully disconnected. Think of a disconnected system as a system that always is or sometimes is on airplane mode.

You bring this single file (or package) with you to the system you want to install or update new software onto. The package includes instructions on how to assemble all the pieces of software (components) once on the other side. These instructions are fully "declarative," which means that everything is represented by code and automated vs manual. The hardest part is assembling the declarative package on the connected side. But once everything is packaged up, Zarf makes even massively complex systems easy to install, update, and maintain within disconnected systems.

Such packages also become highly distributable, as they can now run on edge, embedded systems, secure cloud, data centers, or even on a local environment. This is incredibly helpful for organizations that need to integrate and deploy software from multiple secure development environments from a disparate set of development teams into disconnected IT operational environments. Zarf helps ensure that development teams can integrate with the production environment they are deploying to, even if they will never actually touch that environment.

Zarf makes DevSecOps for air gap possible.

Zarf is written entirely in go, except for a single 400Kb binary for the injector sytem written in rust so we can fit it in a configmap. All assets are bundled together into a single zstd tarball on each zarf package create operation. On the air gap / offline side zarf package deploy extracts the various assets and places them on the filesystem or installs them in the cluster, depending on what the zarf package says to do. Some important ideas behind zarf:

• All workloads are installed in the cluster via the Helm SDK
• The OCI Registries used are both from Docker
• Currently the Registry and Git servers are not HA, see #375 and #376 for discussion on this
• In order to avoid TLS issues, Zarf binds to 127.0.0.1:31999 on each node as a NodePort to allow all nodes to access the pod(s) in the cluster
• Until #306 is merged, during helm install/upgrade a Helm PostRender function is called to mutate images and ImagePullSecrets so the deployed resoures use the NodePort binding
• Zarf uses a custom injector system to bootstrap a new cluster. See the PR #329 and ADR for more details on how we came to this solution. The general steps are listed below:
  • Get a list images in the cluster
  • Attempt to create an ephemeral pod using an image from the lsit
  • A small rust binary that is compiled using musl to keep the size the max binary size of ~ 672 KBs is injected into the pod
  • The mini zarf registry binary and docker:2 images are put in a tar archive and split into 512 KB chunks, larger sizes tended to cause latency issues on low-resource control planes
  • An init container runs the rust binary to reassabmle and extract the zarf binary and registry image
  • The container then starts and runs the zarf binary to host the registry image in an embedded docker registry
  • After this the main docker registry chart is deployed, pulls the image from the ephemeral pod and destroys the created configmaps, pod and service

Source DrawIO