Commit 765c8619 authored by Jakub Ružička's avatar Jakub Ružička
Browse files

new apkg documentation

  d8888b.  .d88b.   .o88b. .d8888.
  88  `8D .8P  Y8. d8P  Y8 88'  YP
  88   88 88    88 8P      `8bo.
  88   88 88    88 8b        `Y8b.
  88  .8D `8b  d8' Y8b  d8 db   8D
  Y8888D'  `Y88P'   `Y88P' `8888Y'
parent 99a1308e
Pipeline #75980 passed with stages
in 3 minutes and 24 seconds
"""
build package
build packages
usage: apkg build ([-u] | [-s <srcpkg>] | [-a <ar>] )
Usage: apkg build ([-u] | [-s <srcpkg>] | [-a <ar>] )
[-v <ver>] [-r <rls>] [-d <distro>]
[-i] [-I] [--no-cache]
options:
Options:
-s <srcpkg>, --srcpkg <srcpkg> use specified source package (path or name)
-a <ar>, --archive <ar> use specified archive (path or name)
-u, --upstream use upstream archive / apkg get-source
......
"""
install build dependencies
usage: apkg build-dep ([-u] | [-s <srcpkg>] | [-a <ar>] )
Usage: apkg build-dep ([-u] | [-s <srcpkg>] | [-a <ar>] )
[-v <ver>] [-r <rls>] [-d <distro>]
[-y]
options:
Options:
-s <srcpkg>, --srcpkg <srcpkg> use specified source package (path or name)
-a <ar>, --archive <ar> use specified archive (path or name)
-u, --upstream use upstream archive / apkg get-source
......
"""
Download upstream archive for current project
download upstream archive for current project
using project.upstream_archive_url config option
usage: apkg get-archive [-v <ver>] [--no-cache]
Usage: apkg get-archive [-v <ver>] [--no-cache]
options:
Options:
-v <ver>, --version <ver> version of archive to download
--no-cache disable cache
""" # noqa
......
"""
Create dev archive from current project state
create dev archive from current project state
using script specified by project.make_archive_script config option
usage: apkg make-archive [-v <ver>] [--no-cache]
Usage: apkg make-archive [-v <ver>] [--no-cache]
options:
Options:
-v <ver>, --version <ver> rename archive to match specified version if needed
--no-cache disable cache
""" # noqa
......
"""
create source package (files to build package from)
usage: apkg srcpkg [-a <ar>] [-u]
Usage: apkg srcpkg [-a <ar>] [-u]
[-v <ver>] [-r <rls>] [-d <distro>]
[--no-cache]
options:
Options:
-u, --upstream use upstream archive / apkg get-source
default: use dev archvie / apkg make-source
-a <ar>, --archive <ar> use specified archive (path or name)
......
"""
show status of current project
usage: apkg status
Usage: apkg status
"""
import os
......
"""
This is 'arch' apkg package style for Arch linux
apkg package style for **Arch** linux.
**source template:** `PKGBUILD`
**source package:** `PKGBUILD`
**packages:** `*.zst` built using `makepkg`
"""
import glob
from pathlib import Path
......
"""
This is 'deb' apkg package style for Debian-based distros such as
Debian, Ubuntu, and their many clones.
apkg package style for **Debian**
and its many clones such as Ubuntu or Mint.
**source template**: content of `debian/` dir (`control`, `changelog`, ...)
**source package:** `*.dsc` + archives
**packages:** `*.deb` built directly using `dpkg-buildpackage`
or `--isolated` using `pbuilder`
"""
import glob
import os
......
"""
This is 'rpm' apkg package style for RPM-based distros such as
Fedora, CentOS, SUSE, RHEL.
apkg package style for RPM-based distros
such as Fedora, CentOS, SUSE, RHEL.
!!! warning
`rpm` pkgstyle **isn't finished yet**! it will be done for v0.1
**source template**: `*.spec`
!!! TODO
**source package:** `*.src.rpm`
!!! TODO
**packages:** `*.rpm` built directly using `rpmbuild`
or `--isolated` using `mock`
"""
import glob
......
......@@ -23,4 +23,4 @@ RUN apt-get install -y -qqq tox python3-pytest python3-pytest-sugar python3-pyte
RUN apt-get install -y -qqq python3-flake8 pylint3
# doc deps
RUN apt-get install -y -qqq python3-pip
RUN pip3 install mkdocs
RUN pip3 install mkdocs mkdocs-macros-plugin
# apkg - packaging automation tool
# apkg the packaging automation tool
apkg is **going to be** Free and Open Source minimalist cross-distro packaging
automation tool aimed at producing **high quality native packages** for many
different OS distributions/packaging systems with minimum overhead.
**[apkg intro](intro.md)**
## status
**development started recently**
**early development, approaching v0.1**
* [apkg design](design.md)
* [source code](https://gitlab.nic.cz/packaging/apkg)
* [merge requests](https://gitlab.nic.cz/packaging/apkg/-/merge_requests)
* [issues](https://gitlab.nic.cz/packaging/apkg/-/issues)
......@@ -19,89 +19,17 @@ patches, issues, comments and other contributions are welcome **ᕕ( ᐛ )ᕗ**
## installation
`apkg` is going to provide various packages of itself once it's useful, but
for now please use the source.
**[installation docs](install.md)**
### requirements
Python module requirements are listed in `requirements.txt`.
Once you have `pbr` (`python3-pbr`) installed, rest of requirements can be
automatically installed by `setup.py`.
### user development install
I use this for python projects I develop actively:
```
git clone https://gitlab.nic.cz/packaging/apkg
cd apkg
python3 setup.py develop --user
```
This creates a link in `~/.local/lib/python3.X/site-packages/apkg.egg-link`
and also installs `apkg` script into `~/.local/bin` so make sure you have
`~/.local/bin` in your `$PATH`, possibly before system `bin` paths to override
`apkg` scripts provided by system packages.
### virtualenv install
If you don't want to taint your system by `develop` method above and/or you
want `apkg` isolated from the rest of your system, you can use `virtualenv`
(`python-virtualenv` package on most distros):
```
git clone https://gitlab.nic.cz/packaging/apkg
cd apkg
virtualenv venv
source venv/bin/activate
python3 setup.py develop
apkg
```
You can enter the `venv` later/from other terminals by
```
source apkg/venv/bin/activate
```
You can also do this little trick inside `venv` to avoid conflicts with system `apkg`
packages (assuming `~/bin` is in your `$PATH`):
```
source apkg/venv/bin/activate
ln `which apkg` ~/bin/apkg-dev
apkg-dev
```
## usage
Simply run
**[packaging guide](guide.md)**
```
apkg
```
to get summary of available commands (equivalent of `apkg --help`).
Use `--help`/`-h` after a command to get help for that particular command instead:
```
apkg status --help
```
## contact
## bug reports, feature requests, contact
Use [gitlab issues](https://gitlab.nic.cz/packaging/apkg/-/issues)
to communicate anything you have in mind. Feedback is most appreciated!
## planned features
to communicate anything you have in mind including bugs and improvements.
* tests & CI
* more docs
* packaging
* much more
Feedback is most appreciated!
"""
generate docs from apkg code/docstrings using mkdocs-macros-plugin
"""
from apkg import pkgstyle
from apkg.cli import cmd2mod
from pathlib import Path
BASE_PATH = Path(__file__).parent.parent
BASE_CODE_URL = "https://gitlab.nic.cz/packaging/apkg/-/blob/master/"
APKG_NEW_ISSUE_URL = "https://gitlab.nic.cz/packaging/apkg/-/issues/new"
def define_env(env):
"""
this is available in docs using jinja2 templates
"""
env.variables.pkgstyles = pkgstyle.PKGSTYLES
env.variables.new_issue_url = APKG_NEW_ISSUE_URL
@env.filter
def relpath(path):
return Path(path).relative_to(BASE_PATH)
@env.filter
def file_link(path):
return "[{fn}]({url}{fn})".format(
fn=path,
url=BASE_CODE_URL)
@env.filter
def file_text(path):
text = Path(path).open('r').read().strip()
return "``` text\n%s\n```" % text
@env.filter
def mod_doc(modname):
mod = __import__(modname, fromlist=[''])
return mod.__doc__.strip()
@env.filter
def cmd_help(cmd):
modname = 'apkg.commands.%s' % cmd2mod(cmd)
return "``` text\n$> apkg %s --help\n\n%s\n```" % (
cmd, mod_doc(modname))
# apkg commands
To get a summary of available `apkg` commands, simply run it without parameters:
``` text
$> apkg
{{ 'apkg.cli' | mod_doc }}
```
Use `--help`/`-h` after a command to get help for that particular command instead:
``` text
$> apkg command -h
```
## status
{{ 'status' | cmd_help }}
Example:
``` text
$> apkg status
project name: apkg
project base path: /home/u/src/apkg
project VCS: git
project config: distro/config/apkg.toml (exists)
package templates path: distro/pkg (exists)
package templates:
arch: distro/pkg/arch
deb: distro/pkg/deb
rpm: distro/pkg/rpm
current distro: arch / Arch Linux
package style: arch
package template: distro/pkg/arch
```
## make-archive
{{ 'make-archive' | cmd_help }}
`make-archive` requires
[project.make_archive_script](config.md#projectmake_archive_script)
config option to be set.
This command will only succeed when the script finishes successfully (with exit code 0) and the resulting archive it created and printed to last line of its stdout exists, otherwise it will complain accordingly.
Resulting archive is copied to `pkg/archives/dev/`.
## get-archive
{{ 'get-archive' | cmd_help }}
`get-archive` requires
[project.upstream_archive_url](config.md#projectmake_archive_script)
config option to be set.
This command will only succeed when it managed to download specified archive.
Archive is downloaded into `pkg/archives/upstream/`.
!!! notice
automatic latest version detection isn't implemented yet so version needs
to be manually specified with `-v`/`--version` option for now, i.e.
`apkg get-archive -v 1.2.3`
## srcpkg
{{ 'srcpkg' | cmd_help }}
!!! TODO
this and all following commands require proper explanation
## build
{{ 'build' | cmd_help }}
## build-dep
{{ 'build-dep' | cmd_help }}
# apkg config
## project config
`apkg` looks for project config file `distro/config/apkg.toml`.
Please see {{ 'distro/config/apkg.toml' | file_link }} for up-to-date example.
Currently, all options belong under `[project]` section:
### project.name
By default, project name is guessed from project dir name but that can break easily when the project dir is renamed so it's better to specify it explicitly in using `name`
```
[project]
name = "banana"
```
This is available to templates through {% raw %}`{{ project.name }}`{% endraw %}.
### project.make_archive_script
In order to create packages from your project source, `apkg` requires a script which creates archive from current project state.
The script MUST return the path to created archive on last line of its `stdout`.
Include such script in your project and then point to it using `make_archive_script`:
```
[project]
make_archive_script = "scripts/make-dev-archive.sh"
```
### project.upstream_archive_url
To easily download upstream archives using `apkg` you can specify `upstream_archive_url` with templating available:
{% raw %}
```
[project]
upstream_archive_url = "https://banana.proj/dl/{{ project.name }}/{{ project.name }}-{{ version }}.tar.xz"
```
{% endraw %}
# apkg packaging guide
This guide assumes you have:
* read [apkg intro](intro.md)
* [installed apkg](install.md)
## project setup
All `apkg` input files reside in top-level `distro/` directory by convention.
In order to use `apkg` in your project you need to provide it with:
* project metadata stored in `distro/config/apkg.toml`
* package templates in individual directories of `distro/pkg/`
Let's start by entering top level project dir and creating `distro/` there:
``` shell
cd project
mkdir distro
```
### project config - apkg.toml
Let's create `distro/config` directory and open new `apkg.toml` in your favorite editor:
``` shell
mkdir distro/config
edit distro/config/apkg.toml
```
You can use `apkg`'s {{ 'distro/config/apkg.toml' | file_link }} as a starting point.
See [project config docs](config.md#project-config) for a
complete list of individual `apkg.toml` options including descriptions.
This guide assumes you have following options specified in your `apkg.toml`:
* [project.name](config.md#projectname)
* [project.make_archive_script](config.md#projectmake_archive_script)
* [project.upstream_archive_url](config.md#projectupstream_archive_url) if project has upstream archives
Confirm that `apkg status` in project directory mentions existing config file:
``` text
$> apkg status | grep 'project config'
project config: distro/config/apkg.toml (exists)
```
Test `project.make_archive_script` option using `apkg make-archive`:
``` text
$> apkg make-archive
I creating dev archive
I running make_archive_script: scripts/make-dev-archive.sh
I archive created: dist/apkg-0.0.2.tar.gz
I copying archive to: pkg/archives/dev/apkg-0.0.2.tar.gz
✓ made archive: pkg/archives/dev/apkg-0.0.2.tar.gz
pkg/archives/dev/apkg-0.0.2.tar.gz
```
If you run into issues, consider appending `--verbose` or `--debug` to `apkg` command in question to print more detailed information.
!!! TODO
`apkg make-archive` isn't finished yet, include it here once it is.
Great, you're now able to create archives required to create packages!
### package templates
Next we need to create individual [package templates](templates.md) to contain all files needed to create source package using one of supported [packaging styles](pkgstyles.md).
Each directory in `distro/pkg/` is considered a template.
{% raw %}
Version string should be replaced with `{{ version }}` macro in relevant files and such templating is available for all files present in a template - you can reference ``{{ project.name }}`` and more.
{% endraw %}
This is best demonstrated on `apkg` itself:
* `arch` template: {{ 'distro/pkg/arch' | file_link }}
* `deb` template: {{ 'distro/pkg/deb' | file_link }}
{% raw %}
!!! Note
`apkg` doesn't provide means to create new templates as that's handled on
distro level. Just use standard way of creating packages on the target platform
and put the resulting packaging source files into template dir and adjust
`{{ version }}`. Use target distro's packaging docs and use similar
packages already in distro repos as a reference.
{% endraw %}
Please consult [package template docs](templates.md) alongside target distro
packaging docs and you should eventually arrive at `apkg status` mentioning
newly created package templates:
``` text
$> apkg status
project name: project
project base path: /home/u/src/project
project VCS: git
project config: distro/config/apkg.toml (exists)
package templates path: distro/pkg (exists)
package templates:
arch: distro/pkg/arch
deb: distro/pkg/deb
current distro: arch / Arch Linux
package style: arch
package template: distro/pkg/arch
```
## apkg workflow overview
``` text
apkg packaging workflow
+------------------------------+ +------------------------------------+
| | | |
| $ apkg make-archive | | $ apkg get-archive [-v 1.2.3] |
| | | |
| in: current project state | OR | in: archive hosted online |
| | | |
| out: pkg/archives/dev/*.xz | | out: pkg/archives/upstream/*.xz |
| | | |
+---------------+--------------+ +----------------+-------------------+
| |
| |
| |
v v
+----------+------------------------------------+-------------+
| |
| $ apkg srcpkg |
| |
| in: distro/pkg/$TEMPLATE/ (package template) |
| pkg/archives/*/*.xz (archive) |
| |
| out: pkg/srcpkgs/$DISTRO/$SRCPKG (source package) |
| pkg/build/srcpkgs/$DISTRO/$SRCPKG/ (build dir) |
| |
+------------------------------+------------------------------+
|
|
|
v
+------------------------------+------------------------------+
| |
| $ apkg build |
| |
| in: pkg/srcpkgs/$DISTRO/$SRCPKG (source package) |
| |
| out: pkg/pkgs/$DISTRO/$PKG (package) |
| pkg/build/pkgs/$DISTRO/$PKG (build dir) |
| |
+-------------------------------------------------------------+
```
`srcpkg` uses `get-archive` to create `dev` archive by default but it can
also download upstream archive using `get-archive` when `--upstream` is
passed or use local archive directly using `--archive`.
Similarly, `build` uses `srcpkg` to create source package by default but it
can be directly specified using `--srcpkg` option.
This provides convenience of high level commands reusing lower level ones as
needed but also flexibility to run each individual step manually.
## usage
To get a summary of available [commands](commands.md) simply run `apkg` without parameters.
Use `--help`/`-h` after a command to get help for that particular command instead:
``` text
$> apkg build -h
```
Detailed description of each command is available in [commands docs](commands.md).
You can control `apkg` output format and verbosity using following global options:
* `--debug`: print everything; include source file, line number, and function name
* `--verbose`: print more things; include module name and function
* `--brief`: only print important things like success, errors, and command output
* `--quiet`: suppress all logging output, only print command results
!!! tip
`--verbose` and `--debug` options can be **very helpful** when debugging, try adding one to your failing `apkg` command to gain more insight.
## build packages
`apkg build` is a primary `apkg` command used to build packages that should
cover vast majority of use cases.
Try running `apkg build` and see what it says. `apkg` should explain clearly
if something is wrong, if that's not the case please do open a [new issue]({{
new_issue_url }}) as that's a serious usability problem.