Skip to content

Drone Pipelines

Summary

Every Aetos repository ships a Drone CI definition at .drone.yml. There is exactly one standard per repository, and there are three types of file, chosen by what the repository produces:

Type File Produces Repos
A — PyPI package templates/drone/python-lib.yaml A wheel + sdist on the private index dagpipe, doc-forge, hexa, mail-intake, mongo-ops, omniread, openapi-first, py-jwt
B — npm package templates/drone/npm-lib.yaml A package published to the private npm registry doc-weaver, react-auth, react-openapi
C — Docker deployment templates/drone/service-docker.yaml An image pushed to the registry and run as a container auth-server

All templates run on platform: linux / arm64 with the workspace at /drone/src. Copy the matching template to <repo>/.drone.yml; the Type A and Type B files need no edits, the Type C file needs its placeholders filled in (below).

Conventions: .drone.yml (not .yaml), one file per repo, secrets injected with from_secret — never commit credentials.

Type A — PyPI package (python-lib.yaml)

For internal Python libraries. The file contains two pipelines:

  1. build-and-publish-pypi — triggered on tag:
  2. check-version — guard: fail unless $DRONE_TAG equals the version in pyproject.toml; then read name / version and query $PIP_REPO_URL/simple/<package>/; if that version already exists, exit 78 and skip the whole build (see 91 – Versioning & Changelog).
  3. quality-gate — install the package with its [dev] extras from the private index, then black --check, ruff check, mypy, pydoclint, pytest.
  4. build-package — python -m build into dist/.
  5. upload-to-private-pypi — twine upload to $PIP_REPO_URL.
  6. backfill-pypi-from-tags — triggered on custom:
  7. fetch-tags — git fetch --tags --force.
  8. build-and-upload-missing — walk every tag in version order, skip the ones already on the index, git checkout the rest, verify tag == pyproject version at that commit, build, and upload. This is the disaster-recovery / replay path.

Secrets: PIP_REPO_URL, PIP_USERNAME, PIP_PASSWORD.

  • Ship as-is. Package name and version come from pyproject.toml, so the same file is reused verbatim across every library.

Type B — npm package (npm-lib.yaml)

For internal npm libraries (the React packages). The file contains two pipelines:

  1. build-and-publish-npm — triggered on tag:
  2. check-version — guard: fail unless $DRONE_TAG equals the version in package.json; query $NPM_REPO_URL for name@version; if it already exists, exit 78 and skip the whole build.
  3. quality-gate — write .npmrc from the secrets, npm install, then npm run quality-gate.
  4. build-package — npm install && npm run build into dist/.
  5. publish-to-private-npm — npm publish --access restricted to $NPM_REPO_URL.
  6. backfill-npm-from-tags — triggered on custom:
  7. fetch-tags — git fetch --tags --force.
  8. build-and-publish-missing — walk every tag in version order, skip the ones already on the registry, git checkout the rest, verify tag == package.json version at that commit, build, and publish. This is the disaster-recovery / replay path.

Secrets: NPM_REPO_URL, NPM_USERNAME, NPM_PASSWORD. Auth is a generated .npmrc written from secrets at runtime. It sets a scope-scoped registry, never a global one — @aetoskia:registry=$NPM_REPO_URL — plus a host-scoped auth line //<host>/:_auth=<base64(user:pass)>. The scope and host are derived from package.json and NPM_REPO_URL respectively. A global registry= would push public transitive deps through Verdaccio (which has no uplink) and 404 npm install; a bare _auth= is rejected by modern npm. See 93 – Library Registry.

  • Ship as-is. Name and version come from package.json, so the same file is reused verbatim across every npm package.

Registry credentials are shared. The private npm registry (Verdaccio at npm.aetoskia.com) authenticates against the same htpasswd file as the private PyPI index and the Docker registry (/auth/htpasswd), with registration disabled (max_users: -1). NPM_USERNAME / NPM_PASSWORD are therefore the same credentials as PIP_USERNAME / PIP_PASSWORD — no separate npm account exists. Verdaccio's htpasswd plugin accepts bcrypt hashes ($2y$…); non-bcrypt entries that pypiserver tolerates will not authenticate against npm. See 93 – Library Registry.

Type C — Docker deployment (service-docker.yaml)

For a service that ships a container image — a long-running API/worker, e.g. auth-server. The file contains one pipeline, build-and-deploy-service, triggered on tag:

  • fetch-tags — resolve the latest tag into LATEST_TAG.txt; fail if there are no tags.
  • check-version — guard: fail unless $DRONE_TAG equals the repo-root VERSION file (see 91 – Versioning & Changelog).
  • check-remote-image — if aetos/<app>:<tag> already exists locally, exit 78 and skip the build.
  • quality-gate — install from requirements.txt and run black / ruff / mypy / pytest.
  • build-image — docker build with the private-index build args, tagged aetos/<app>:<tag> and :latest.
  • push-image — login, re-tag with $REGISTRY_HOST, push both tags.
  • stop-old — docker rm -f <app>.
  • run-container — docker run -d --name <app> -p <host>:<container> --restart always with the service's -e environment.

Secrets: PIP_REPO_URL, PIP_USERNAME, PIP_PASSWORD (service build args), REGISTRY_HOST, REGISTRY_USER, REGISTRY_PASS, plus the service runtime env (for example MONGO_*, JWT_SECRET).

Placeholders to fill in per repo:

Placeholder Meaning Example
<app> service image / container name auth-server
aetos/<app> image repository on the registry aetos/auth-server
<host-port> / <container-port> published : in-image port 9003:8000
<ENV_VAR> runtime environment variable MONGO_HOST

What not to do

  • Do not hand-roll a fourth shape. A repo is exactly one of Type A, B, or C, never combinations.
  • Do not inline secrets; always use from_secret.
  • Do not drop the check-version / check-remote-image skip — it guards the index and the running container against duplicate publishes.
  • Do not drop the tag↔version guard. A Python package needs its pyproject.toml version to equal the tag; an npm package needs its package.json version; a service needs a root VERSION file. See 91 – Versioning & Changelog.
  • Do not rename the pipelines; tooling and runbooks key off build-and-publish-pypi, build-and-publish-npm, build-and-deploy-service, and their backfill-*-from-tags companions.