315 lines
17 KiB
HTML
Generated
315 lines
17 KiB
HTML
Generated
<!DOCTYPE html>
|
||
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="generator" content="pandoc" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
|
||
<meta name="author" content="Juno Takano" />
|
||
<meta name="dcterms.date" content="2024-06-30" />
|
||
<title>Introducing tori • jutty.dev</title>
|
||
<style>
|
||
code{white-space: pre-wrap;}
|
||
span.smallcaps{font-variant: small-caps;}
|
||
div.columns{display: flex; gap: min(4vw, 1.5em);}
|
||
div.column{flex: auto; overflow-x: auto;}
|
||
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
||
/* The extra [class] is a hack that increases specificity enough to
|
||
override a similar rule in reveal.js */
|
||
ul.task-list[class]{list-style: none;}
|
||
ul.task-list li input[type="checkbox"] {
|
||
font-size: inherit;
|
||
width: 0.8em;
|
||
margin: 0 0.8em 0.2em -1.6em;
|
||
vertical-align: middle;
|
||
}
|
||
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
|
||
/* CSS for syntax highlighting */
|
||
pre > code.sourceCode { white-space: pre; position: relative; }
|
||
pre > code.sourceCode > span { line-height: 1.25; }
|
||
pre > code.sourceCode > span:empty { height: 1.2em; }
|
||
.sourceCode { overflow: visible; }
|
||
code.sourceCode > span { color: inherit; text-decoration: inherit; }
|
||
div.sourceCode { margin: 1em 0; }
|
||
pre.sourceCode { margin: 0; }
|
||
@media screen {
|
||
div.sourceCode { overflow: auto; }
|
||
}
|
||
@media print {
|
||
pre > code.sourceCode { white-space: pre-wrap; }
|
||
pre > code.sourceCode > span { display: inline-block; text-indent: -5em; padding-left: 5em; }
|
||
}
|
||
pre.numberSource code
|
||
{ counter-reset: source-line 0; }
|
||
pre.numberSource code > span
|
||
{ position: relative; left: -4em; counter-increment: source-line; }
|
||
pre.numberSource code > span > a:first-child::before
|
||
{ content: counter(source-line);
|
||
position: relative; left: -1em; text-align: right; vertical-align: baseline;
|
||
border: none; display: inline-block;
|
||
-webkit-touch-callout: none; -webkit-user-select: none;
|
||
-khtml-user-select: none; -moz-user-select: none;
|
||
-ms-user-select: none; user-select: none;
|
||
padding: 0 4px; width: 4em;
|
||
}
|
||
pre.numberSource { margin-left: 3em; padding-left: 4px; }
|
||
div.sourceCode
|
||
{ color: #cccccc; background-color: #303030; }
|
||
@media screen {
|
||
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
|
||
}
|
||
code span.al { color: #ffcfaf; } /* Alert */
|
||
code span.an { color: #7f9f7f; font-weight: bold; } /* Annotation */
|
||
code span.at { } /* Attribute */
|
||
code span.bn { color: #dca3a3; } /* BaseN */
|
||
code span.bu { } /* BuiltIn */
|
||
code span.cf { color: #f0dfaf; } /* ControlFlow */
|
||
code span.ch { color: #dca3a3; } /* Char */
|
||
code span.cn { color: #dca3a3; font-weight: bold; } /* Constant */
|
||
code span.co { color: #7f9f7f; } /* Comment */
|
||
code span.cv { color: #7f9f7f; font-weight: bold; } /* CommentVar */
|
||
code span.do { color: #7f9f7f; } /* Documentation */
|
||
code span.dt { color: #dfdfbf; } /* DataType */
|
||
code span.dv { color: #dcdccc; } /* DecVal */
|
||
code span.er { color: #c3bf9f; } /* Error */
|
||
code span.ex { } /* Extension */
|
||
code span.fl { color: #c0bed1; } /* Float */
|
||
code span.fu { color: #efef8f; } /* Function */
|
||
code span.im { } /* Import */
|
||
code span.in { color: #7f9f7f; font-weight: bold; } /* Information */
|
||
code span.kw { color: #f0dfaf; } /* Keyword */
|
||
code span.op { color: #f0efd0; } /* Operator */
|
||
code span.ot { color: #efef8f; } /* Other */
|
||
code span.pp { color: #ffcfaf; font-weight: bold; } /* Preprocessor */
|
||
code span.sc { color: #dca3a3; } /* SpecialChar */
|
||
code span.ss { color: #cc9393; } /* SpecialString */
|
||
code span.st { color: #cc9393; } /* String */
|
||
code span.va { } /* Variable */
|
||
code span.vs { color: #cc9393; } /* VerbatimString */
|
||
code span.wa { color: #7f9f7f; font-weight: bold; } /* Warning */
|
||
</style>
|
||
<link rel="stylesheet" href="../assets/css/style.css" />
|
||
<link rel="alternate" type="application/rss+xml" title="RSS (English)" href="https://blog.jutty.dev/assets/rss/en.rss"/>
|
||
<link rel="alternate" type="application/rss+xml" title="RSS (Português)" href="https://blog.jutty.dev/assets/rss/pt.rss"/>
|
||
<link rel="icon" type="image/ico" href="../assets/img/favicon.ico">
|
||
<script src="../assets/js/post-l10n.js" defer></script>
|
||
</head>
|
||
<body>
|
||
<header id="title-block-header">
|
||
<div id="title-container">
|
||
<h1 class="title">Introducing tori</h1>
|
||
</div>
|
||
</header>
|
||
<main class="article">
|
||
<p><strong>tori</strong> is a configuration management and system
|
||
replication tool being rewritten with portability and idempotency in
|
||
mind.</p>
|
||
<p>For the past 5 months, I’ve been simultaneously using and writing it
|
||
to manage my main machine’s configuration. By “manage the configuration”
|
||
what I mean is keeping track of installed packages, configuration files,
|
||
symlinks, and other settings that can be queried and set through command
|
||
line interfaces.</p>
|
||
<p>After installing a given system, I wanted to get it to the same
|
||
configuration state I was used to, or to a certain configuration
|
||
specific to its purpose. Just copying backups would certainly be a very
|
||
manual task, namely because:</p>
|
||
<ul>
|
||
<li>Not all settings live in <code>/etc</code></li>
|
||
<li>Some settings must be set using a specific CLI utility</li>
|
||
<li>Backups usually carry an overwhelming amount of redundant default
|
||
configuration you never even touched</li>
|
||
<li>It does not track what is changing as you are still using the
|
||
system</li>
|
||
<li>I actually wanted to <em>know</em> what I was tracking</li>
|
||
</ul>
|
||
<p>Configuring a system can become a very vague process as you start to
|
||
lose track of where the changes are being made and what is the specific
|
||
configuration needed for something to work.</p>
|
||
<p>Every time you change some configuration file, every time you create
|
||
a symlink somewhere, that’s all having effects on the system that you
|
||
may expect to be there in the future, but you may not remember how to
|
||
accomplish that. This drift between what you have and what you are able
|
||
to replicate only grows as you keep using your system.</p>
|
||
<p>To get a better idea, see the code snippet below. It’s from the main
|
||
file that I use to manage all function calls:</p>
|
||
<div class="sourceCode" id="cb1"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a> <span class="ex">xbps_get_many</span> packages</span>
|
||
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a></span>
|
||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a> <span class="ex">check_service</span> dbus</span>
|
||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a> <span class="ex">check_group</span> audio</span>
|
||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a> <span class="ex">check_group</span> video</span>
|
||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
|
||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> <span class="ex">copy</span> dhcpcd.conf /etc/dhcpcd.conf</span>
|
||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> <span class="ex">place</span> kernel-cmd-line.conf /etc/dracut.conf.d</span>
|
||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> <span class="ex">check_link</span> /etc/localtime /usr/share/zoneinfo/America/Sao_Paulo</span>
|
||
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a></span>
|
||
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="ex">nix_get</span> tailspin tspin</span>
|
||
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="ex">cargo_get</span> taplo-cli taplo <span class="at">--locked</span></span>
|
||
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="ex">bun_get</span> bash-language-server</span>
|
||
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> <span class="ex">get_bin</span> <span class="st">'https://raw.githubusercontent.com/hackerb9/lsix/master/lsix'</span> lsix</span>
|
||
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a></span>
|
||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> <span class="ex">check_gsettings</span> gtk-theme <span class="st">'Plata-Noir'</span></span>
|
||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> <span class="ex">check_gsettings</span> font-name <span class="st">'Mononoki Nerd Font Regular'</span></span></code></pre></div>
|
||
<p>What is happening here:</p>
|
||
<ol type="1">
|
||
<li>A file named <code>packages</code> containing package names is
|
||
parsed and all packages are queried and installed by the
|
||
<code>xbps</code> package manager, if not already installed</li>
|
||
<li>Service <code>dbus</code> is enabled if not already enabled</li>
|
||
<li>The user is added to groups <code>audio</code> and
|
||
<code>video</code>, unless already in them</li>
|
||
<li>File <code>dhcpcd.conf</code> from the configuration directory’s
|
||
<code>base</code> directory is checked against the one in the passed
|
||
path and overwrites it if the user chooses to do so</li>
|
||
<li>File <code>kernel-cmd-line.conf</code> from the configuration
|
||
directory’s <code>base</code> directory is copied into the passed path.
|
||
If the file already exists or differs, tori will present an error</li>
|
||
<li>A symlink is checked to be on <code>/etc/localtime</code> pointing
|
||
to the passed path. If it doesn’t, it is created or fixed</li>
|
||
<li>If not installed, a few packages are installed using different
|
||
package managers: <code>tailspin</code>, <code>taplo</code> and
|
||
<code>bash-language-server</code></li>
|
||
<li>If absent, an executable for <code>lsix</code> is downloaded from a
|
||
URL and placed at <code>~/.local/bin</code></li>
|
||
<li>Some <code>gsettings</code> values are read and set if they differ:
|
||
<code>gtk-theme</code> and <code>font-name</code></li>
|
||
</ol>
|
||
<p>Notice how everything is conditioned to the system not already
|
||
presenting that state? tori aims to be idempotent. Running it twice
|
||
should do nothing the second time it runs so you can run it multiple
|
||
times while making changes without any doubled effects.</p>
|
||
<p>I mentioned a <code>base</code> directory. This is what a sample tori
|
||
directory would look like in its present state:</p>
|
||
<pre><code>.
|
||
├── base
|
||
│ ├── dhcpcd.conf
|
||
│ ├── kernel-cmd-line.conf
|
||
│ ├── packages
|
||
│ └── vars.sh
|
||
├── .bkp
|
||
│ ├── canonical
|
||
│ │ ├── etc
|
||
│ │ └── opt
|
||
│ └── ephemeral
|
||
│ └── etc
|
||
├── src
|
||
│ ├── checks.sh
|
||
│ ├── copy.sh
|
||
│ └── get.sh
|
||
└── strap</code></pre>
|
||
<p>What you are seeing in this sample of the directory are the following
|
||
files and directories:</p>
|
||
<ul>
|
||
<li><code>base</code>: Where you place the configuration files that
|
||
functions like <code>copy</code> and <code>place</code> will look for
|
||
and copy into the desired locations</li>
|
||
<li><code>.bkp/canonical</code>: Where tori will look for initial
|
||
backups and create them if none exists</li>
|
||
<li><code>.bkp/ephemeral</code>: Where tori will place timestamped
|
||
backups every time a file is modified or overwritten</li>
|
||
<li><code>src</code>: Where the source files live, mostly containing
|
||
function definitions</li>
|
||
<li><code>strap</code>: The main file used to call the functions</li>
|
||
</ul>
|
||
<p>Because I developed tori for my own purposes initially, I didn’t
|
||
really care to separate the actual source files from the
|
||
context-sensitive data. While a mistake from a higher level, it allowed
|
||
me to just keep developing the whole system configuration and the code
|
||
that tracked it from a single, version-controlled location, amounting to
|
||
very little complexity. I can’t deny to have enjoyed it so far, but
|
||
going forward that is going to change.</p>
|
||
<p>Currently, tori is able to install several package managers and their
|
||
packages, including xbps, apt, nix, opam, stack, cargo, go, sdkmanager,
|
||
npm, flatpak and pipx.</p>
|
||
<p>It can also perform several other tasks:</p>
|
||
<ul>
|
||
<li>setup programming runtimes for OCaml, Scala (via Coursier), Go,
|
||
JavaScript (node and bun), Rust (via rustup)</li>
|
||
<li>generate GPG certificates</li>
|
||
<li>query and set options with <code>update-alternatives</code> and
|
||
<code>gsettings</code></li>
|
||
<li>change the user shell</li>
|
||
<li>check and enable services (systemd and runit)</li>
|
||
<li>download pre-built binaries from tarballs and (g)zip files,
|
||
unpacking and making them executable</li>
|
||
<li>get files through the network using rsync</li>
|
||
<li>several other things likely not worth mentioning</li>
|
||
</ul>
|
||
<p>The application slowly grew to accommodate many of my needs, but I
|
||
also made it very hard to share with the world in the process, since I
|
||
never really meant to go public with it.</p>
|
||
<h2 id="portability-issues">Portability issues</h2>
|
||
<p>Despite it being very useful to me in its current state and still
|
||
being something I actively use every day, a lot of it is hard-coded for
|
||
my very personal use. It was not written with portability in mind and
|
||
therefore requires a lot of source-code editing to use in a different
|
||
system.</p>
|
||
<p>For instance, when I switched from Debian to Void Linux, most of it
|
||
broke. I certainly would not expect the package list to be compatible
|
||
between them, but I realized at that point how tightly it was coupled to
|
||
Debian.</p>
|
||
<p>When I started delving deeper into FreeBSD and setting up the system,
|
||
I kept reaching out to something like tori. But it wasn’t there, and it
|
||
wouldn’t work even if it were.</p>
|
||
<p>Something that certainly influenced my desire to write tori was my
|
||
experience using NixOS, which was full of mixed feelings, but undeniably
|
||
had good feelings that stuck.</p>
|
||
<p>I really liked being able to manage the system configuration and
|
||
packages from a single file. But, at the same time, I felt it was
|
||
overkill. It was limiting because most of the time you were
|
||
<strong>forced</strong> to configure things through its interfaces. It
|
||
was basically incompatible with what every other Unix system expects,
|
||
and therefore what people who write software for these systems also
|
||
expect.</p>
|
||
<p>I appreciated bringing the system configuration to a centralized
|
||
file, but I certainly did not want to manage all my
|
||
<code>~/.config</code> configuration files from that same place. After
|
||
writing tori, I can choose what to place under its tracking and what not
|
||
to. Third-party software still works as both me and its creators expect
|
||
it to, instead of my system breaking things and needing them to work the
|
||
way <em>it</em> expects.</p>
|
||
<h2 id="glad-to-reinvent-the-wheel">Glad to reinvent the wheel</h2>
|
||
<p>While I understand there are very mature and powerful tools to manage
|
||
a system’s state and reproduce it, I am aiming here for a much simpler
|
||
use case. I have no intention to see it used in enterprise or
|
||
distributed systems. It is all about managing how your personal
|
||
computing is set up and having a backup of how you did it.</p>
|
||
<p>What I need is not a tool that can orchestrate a fleet of containers
|
||
running a given configuration. What I need is a tool that can run in a
|
||
bare system that just got installed and get it to a state that feels
|
||
useful to me. I do not want it to run instructions over a range of IPs,
|
||
I just want to be able to check at any time if the system state has
|
||
diverged from the configuration I am using to track it. I wanted a tool
|
||
that would help me develop a different habit when I need to make
|
||
system-level changes.</p>
|
||
<p>And finally, I suppose I just really wanted to build this. I really
|
||
enjoy the process of configuring operating systems and learning how they
|
||
work and differ. And I really wanted to learn more about portable,
|
||
POSIX-compatible shell scripting.</p>
|
||
<p>So I decided to rewrite it with portability in mind. I am doing this
|
||
rewrite in FreeBSD, to put the portability to the test. Once some basic
|
||
functionality is done, the next step will be bringing it to Void Linux,
|
||
Debian and NetBSD.</p>
|
||
<p>tori is a bird that has just hatched, so everything is still very,
|
||
very crude. At this stage, the docs often show intentions rather than
|
||
implemented functionality. Still, because it is something I’ve come to
|
||
depend on, it has this rewarding sense of usefulness behind it.</p>
|
||
<p>If it sounds interesting to you, take a look. You can follow
|
||
development at the main <a href="https://brew.bsd.cafe/jutty/tori">Git
|
||
repository</a> in BSD.Cafe’s Git forge or through the <a
|
||
href="https://github.com/jultty/tori">GitHub mirror</a>. Going forward,
|
||
I will also probably be talking a lot about it on my <a
|
||
href="https://mastodon.bsd.cafe/@jutty">Mastodon profile</a>.</p>
|
||
</main class="article">
|
||
</body>
|
||
<footer>
|
||
<hr/>
|
||
<div class="footer-text">
|
||
<span id="author-pre-text">posted by</span>
|
||
<span class="author">Juno Takano</span>
|
||
<span id="date-pre-text">on</span>
|
||
<span class="date">June 30, 2024</span>
|
||
<span class="footer-back" aria-role="nav" aria-label="Navigate back">
|
||
↩ <a id="footer-back-link" href="../index.html">Back</a>
|
||
</span>
|
||
</div>
|
||
</footer>
|
||
</html>
|