Quick start 'lk' command
This tutorial shows how points are created and viewed using the cli (command line instruction) `lk`. Initially, we write points into a file and then back. In the second part we illustrate the use of a database and the commands to monitor its content. And finally, we show how the linkspace command `lk` can create small apps. Use a programming language as e.g. Python when doing non-trivial stuff as explained in another tutorial. However, when programming, the `lk` terminal command is useful for debugging.
Let's start to see if linkspace is available on this computer.
lk --version
linkspace-cli linkspace-cli - 0.7.0 - main - 3887000f - 1.87.0-nightly - 2025-04-03T06:41:33.359797450Z
1 Point
Linkspace is built around the concept of `point`s. A point holds some header information and a databody just shy of 64kb.
There are three different type of points: the extreme simple and limited datapoint, linkpoint with space, i.e. group-domain-path and zero or more links to other points, and the keypoint with a signed key such that the originator of the point can be verified.
Before beginning, go to the temporary dictory and remove an existing log file named mylog
cd /tmp/ rm mylog
1.1 Datapoint
echo "Hello" | lk datapoint > mylog cat mylog | lk format
## iHzk-ATXbvhKNuZEEjyiyU5_TBJEGxkjxLWAF5ZdL8U ## type DataPoint group [#:pub] domain path pubkey [@:0] stamp 0 links 0 # data 6 # Hello ## end ##
The unique identifier of any point is as above in the first line between the `## …. ##`. It is the hash of a point. The format printout is then followed by the point type and the space trio: group, domain and path. In case of a datapoint these three are public and empty. Also the pubkey, stamp and link are not used. Only the data field is filled.
A datapoint is always in the public group [#:pub] while its domain and path are empty.
By using `[hash:str]` the otherwise unreadable hash string becomes readable. Other formats are 'lk f "[hash/?b]"` or `lk f "[hash]" | xxd`.
echo -n " world" | lk datapoint >> mylog cat mylog | lk format "The hash is: '[hash:str]'"
The hash is: 'iHzk-ATXbvhKNuZEEjyiyU5_TBJEGxkjxLWAF5ZdL8U' The hash is: 'ybsXVj5BrWssHROTN52UtdmSrXaIjdDSQ85Dv2NPOhY'
Without the double quotes the characters "[….]" would be interpreted by the default bash shell. And notice that `lk format` can be shorted to `lk f` as shown below.
1.2 Linkpoint
To create a linkpoint a path (or group or domain other than the pub and local) and/or link is added. If no group and/or domain are specified, the public group [#:pub] is used while domain remain empty.
echo -n some more data | lk linkpoint ::/my/example/spacename >> mylog cat mylog | lk f
## iHzk-ATXbvhKNuZEEjyiyU5_TBJEGxkjxLWAF5ZdL8U ## type DataPoint group [#:pub] domain path pubkey [@:0] stamp 0 links 0 # data 6 # Hello ## end ## ## ybsXVj5BrWssHROTN52UtdmSrXaIjdDSQ85Dv2NPOhY ## type DataPoint group [#:pub] domain path pubkey [@:0] stamp 0 links 0 # data 6 # world ## end ## ## 0famvFf6vfxAqrzieDPlVd2AeEKToD711KTxFojzBF8 ## type LinkPoint group [#:pub] domain path /my/example/spacename pubkey [@:0] stamp 1743666939035061 links 0 # data 0 # ## end ##
A group signals the intended set of recipients. It is 32 bytes long. You might also create your own group with a unique name as shown below.
HELLOGROUP=$(echo "Hello, Linkspacer" | lk datapoint | lk format "[hash:str]") lk linkpoint chat:$HELLOGROUP:/my/contribution | lk format "[group:str]"
EYgfv-KuzF3DjXVpaTxSGIsCktSoOIiJGdWap6fq-vQ
In this example a groupname was used after it was first created by generating a hash string.
Next to [#:pub} there also exist the null, local or private group. This are all zero's `[0;32]` group is referred to by `[#:0]` Similar to IP's localhost, this group is not read/writeable from outside.
In this case the domain `chat` was added. The domain is analogous to a IP port. An application picks a domain name (max 16 bytes). The application should not care in which group it is running.
Finally the path is similar to file-directory structures, written as e.g., "my/contribution".
The domain/group/path combination is also called the space (due to the variable length of a path a space name can be upto 240 bytes long). You can use space statement as e.g., `chat:[#:family-a]:/shoppinglist/saterday*`. But to see all used domain/group/path you can use in using `lk` the ":://*".
Sofar the linkpoint has zero links. A point can have zero or more links to other points. Each point is addressable by its hash. To reference one point from another point, you add a link.
HASH1=$(echo "Hello" | lk datapoint | lk format "[hash:str]") echo $HASH1 HASH2=$(echo " world" | lk datapoint | lk format "[hash:str]") echo $HASH2 lk linkpoint "::/my/example/link" -- "link1:$HASH1" "link2:$HASH2" >> mylog
iHzk-ATXbvhKNuZEEjyiyU5_TBJEGxkjxLWAF5ZdL8U Vx4n_Nkd2LXJHtd5wk6V0UOjiXVFFlUiGGXb2jhc0q0
In the example above first the point hashes are created. In the `lk linkpoint ..` statement a new linkpoint is created a link to each hash. With the mylog of the "Hello" en " world" datapoint and the above linkpoint it is possible to create a dot illustration using a shell script to produce a dot graphviz illustration.
cat mylog | ..../linkspace/apps/run/lk-dot-digraph | dot -Tsvg -o Hello-world.svg
The illustration of the simple dot graphviz
This graphical representation illustrates an example of a more complex mylog
1.3 Keypoint
A keypoint is a linkpoint extended with a signature. First, using a password, create an signature identity with a public and private key part. With that combination, you can sign a point in which case your pubkey in placed in the keypoint.
KEY=$(lk id gen -p 'my secret') lk keypoint ::/my/example/spacename/subspace --sign --lki "$KEY" --password 'my secret' >> mylog cat mylog | lk f
## iHzk-ATXbvhKNuZEEjyiyU5_TBJEGxkjxLWAF5ZdL8U ## type DataPoint group [#:pub] domain path pubkey [@:0] stamp 0 links 0 # data 6 # Hello ## end ## ## ybsXVj5BrWssHROTN52UtdmSrXaIjdDSQ85Dv2NPOhY ## type DataPoint group [#:pub] domain path pubkey [@:0] stamp 0 links 0 # data 6 # world ## end ## ## 0famvFf6vfxAqrzieDPlVd2AeEKToD711KTxFojzBF8 ## type LinkPoint group [#:pub] domain path /my/example/spacename pubkey [@:0] stamp 1743666939035061 links 0 # data 0 # ## end ## ## 3totxCI-qMpmIpDfWVF0PhFyIjNK4hXzhAjtIefg9Us ## type LinkPoint group [#:pub] domain path /my/example/link pubkey [@:0] stamp 1743666939119783 links 2 link1 iHzk-ATXbvhKNuZEEjyiyU5_TBJEGxkjxLWAF5ZdL8U link2 Vx4n_Nkd2LXJHtd5wk6V0UOjiXVFFlUiGGXb2jhc0q0 # data 0 # ## end ## ## YnTWSGdzBtIJ5ppdVJJTRqLAXPHxN8TQZt-esACUQng ## type KeyPoint group [#:pub] domain path /my/example/spacename/subspace pubkey [b:zqlutwR78gTKNxl4rBBxMnWEK_0kU9iLtqbSRjVt8lA] stamp 1743666939574693 links 0 # data 0 # ## end ##
The hashed field, generated with Blake3, at the beginning of the formatted printout is the unique identification of the point. Linkpoints are a special case of keypoints, i.e. th pubkey field in the point header is the pubkey [@:0]. In the above example the pubkey in byte format start with [b:4bgjU…aY]. There is no [@:] format.
There is more to say to key management. For example, you might wonder why the password is used twice. In real life the KEY is generated elsewhere with a pubkey and an encrypted private or secret key, known as pubkey and enckey. Later both are imported and to sign a keypoint the password is needed to generate the secret key from the enckey using the password. This way other can lateron validate with the pubkey that the keypoint was signed by an entity that had the secret key. In other documents, as e.g., the quickstart python document, more is explained key management.
In the quickstart and examples we keep it simple, but try to start using keypoint and keys. For serious application use cryptographic properties from the start. They are difficult to get right afterward.
1.4 Point tools
The point fields are readable by general purpose tools to index, address, and filter the packets. For example, `lk filter query` lists only those with a spacename starting with `::/my/example`. This list only the last generated point in the mylog file.
As the linkpoint in the mylog file is in the local [#:0] group the following command would fail with the error message that the [#:0] can't be used in this context. By setting the LK_PRIVATE parameter to rw also the local/private points can be filtered.
export LK_PRIVATE=rw cat mylog | lk filter query ":://*" | lk format "[path:str] at point [hash:str]"
/my/example/spacename at point 0famvFf6vfxAqrzieDPlVd2AeEKToD711KTxFojzBF8 /my/example/link at point 3totxCI-qMpmIpDfWVF0PhFyIjNK4hXzhAjtIefg9Us /my/example/spacename/subspace at point YnTWSGdzBtIJ5ppdVJJTRqLAXPHxN8TQZt-esACUQng
The point hash, group and pubkeys are 32 bytes. These 32 bytes are encoded in URL-safe, no-padding base64. These are string that are practically unreadable as e.g. [#:pub] is readable version of `j0SWgDvDOLIEny0eEoWgDKWyxpmRmavdvbtXfQYz2bE`.
ABE (ASCII Byte Expression) syntax allows to name and handle to long bytes. Other tools are the ABE tools `lk encode ….` and `lk eval ….` to encode input into ABE or to evaluation an ABE expression. In this context [#:pub], [@:pub], the token # refers to a group, @ to a key. ABE is described also in chapter 4.
2 Database
Python/Rust/Javascript programs create more complex systems and packet flows. Then the use of a linkspace database is common. During program development one can read and monitor the database in terminal session with 'lk' commands.
cd /tmp export LKDB=/tmp/linkspace rm -r linkspace lk init
rm: cannot remove 'linkspace': No such file or directory lmdb "/tmp/linkspace"
When using the `lk` command, you can set a `–write` destinations directly, e.g., to write in the database by `–write /db`.
echo hello world >hello-file lk linkpoint --data-input "hello-file" ::/my/example --write /db lk scan /my/example | lk format lk scan --bare | lk format
## UQtQ0RPcd5buOeAQd4UGR4gcHFyDcgLOYNAQ9WlTlQQ ## type LinkPoint group [#:pub] domain path /my/example pubkey [@:0] stamp 1743666939757329 links 0 # data 12 # hello world ## end ## ## UQtQ0RPcd5buOeAQd4UGR4gcHFyDcgLOYNAQ9WlTlQQ ## type LinkPoint group [#:pub] domain path /my/example pubkey [@:0] stamp 1743666939757329 links 0 # data 12 # hello world ## end ##
The database has three indices. A 'log', 'hash', and the 'tree' index. The log-index is ordered by receive order, the hash-index by the point hash, and the tree-index primarily by a point's [spacename, create stamp].
The database is always accessed through the runtime. The runtime lets multiple processes/threads read, write, and watch for new points.
The library API uses functions as `lks_open()`, `lks_process()`, `lks_save()` as well as callbacks and a user-driven event loops. The bash cli `lk` is focused on piping packets. Commands are `scan` as used above to list each point in the database that satisfy the query. Useful are the `watch` commands as `watch-log`, `watch-tree`, `watch-hash`. These are shorthand for `watch –mode ..`.
For example, enter `lk watch test::/my/ | lk format` and in the other terminal enter a new linkpoint with `lk linkpoint test::/my/test –write /db`.
This time the output should contains `group [#:pub], domain test, and path /my/test`
lk watch /my/ | lk format
Open another terminal:
lk linkpoint -i "hello" ::/my/example --write /db lk linkpoint -i "hello" ::/my/test --write /db
Once a new linkpoint with satisfies the my criteria is process and saved in the database, the watch terminal print the newly arrived point.
When developing an app with stores points in the database, the `lk` commands are handy to monitor what is happening.
3 Applications
Building an application is done by mapping an application state to and from linkspace packets (in the database). For example, a drawing application where multiple people can paint to a shared image board.A simple mapping could be:
- Images data are saved as data points
- Every link in a linkpoint is: a hash to an image, and a tag holding (x,y) coordinates.
Adding an image might look something like:
X=30 ; Y=200 ; IMG="https://upload.wikimedia.org/wikipedia/commons/3/35/Tux.svg" curl -s $IMG | lk datapoint > tux.pkt IMG_HASH=$(cat tux.pkt | lk format "[hash:str]") lk linkpoint imageboard:: -- $(printf "%08d%08d" "$X" "$Y"):$IMG_HASH >> tux.pkt lk save --points ./tux.pkt # Instead of `cat` we can provide a file
You could end with `cat tux.pkt | lk format`, but the image is 50K binary. This would result in a long terminal screen download of the data. Sending the output to a file is more convenient.
Building an image requires the program to watch for new packets in the domain `imageboard::`. On every (new) point it draws it over the image.
lk scan "imageboard::" --max 1 | lk format "[hash:str] has the links:\n [links]"
UHvgsxeWA7_s259MTyMSjR6EOpKoPCZbET-JfBbKjAs has the links: 0000003000000200:CbyP-qmRZK7Zr_G5Tx_a4SXXzR-ZZm7HTJHZOUntqNQ
A link might reference a point that is not (yet) available on device. An application has to decide how to handle the situation. In this example, we'll just wait. Waiting can be done manually. e.g.
lk watch "imageboard::" --max 1 \
| lk format "[links]" \
| cut -d':' -f2 \
| xargs -i lk watch-hash "{}" \
| lk format "got point [hash:str] which has [data_size:str] bytes"
Or use `lk get-links`. It has a few common strategies.
lk watch "imageboard::" --max 1 \
| lk get-links pause \
| lk format "[hash:str]"
To complete the imageboard application we'll have to add a few more steps to merge the data into a single picture. See the examples on doing this and more.
To list the content of the linkspace (database) the `lk scan` command can be written as e.g.,
lk -d ./ scan ":://*" --print-query | lk f lk -d ./ scan --bare --"group=[#:pub]" | lk f
and as many other combinations.
4 What else …
A system to exchange points in a group can be made from scratch. Linkspace does not prescribe a way to do so. Each group / network is different, and no single solution can cover every situation.
Linkspace is designed to only ever be a streams of packets, without additional overhead of a (custom) serialization formats. This was shown in the 'mylog' file we have used thus far. This keeps streams compatible with all tools that process streams.
For example, use `lk watch imageboard:$MYGROUP | …` and forward entire stream to another device using netcat/socat, ssh, email, http, a USB stick, or another way to exchange bytes.
To that end, each packet has a mutable header excluded from the hash.
Filters work on these mutable bytes as well.
This lets you quickly build specific network topologies.
See guide for the mutable field names.
A single linkspace instance can be used by multiple applications on one device, and connect to others. To that end there are some conventions. There are functions that create/watch for point with some predefined spacename, links, and data format.
The `filter` and `watch` commands are syntax sugar over queries. You can add `–print-query` to those commands to see the query used.
Queries are designed such that joining two query strings the result is common subset of both or an error if the union is empty.
lk print-query example::/ok
:LKQ/0.1.0 type 1 [b2:00000101] domain = [a:example] group = [b:WT0_F-AX2-wuZZ_K6Mm9BFmexjWVAoWDOADpfL1wws8] path = /ok
Below an example that goes wrong, on purpose.
lk linkpoint domain:[#:0] | lk save
Creating a packet is ok, but receiving is not accepted with `lk save` as domain [#:0] is private/local domain. By setting the exporting the LK_PRIVATE=rw environment parameter one can deal with points in the local domain too.
4.1 Environment parameters
fs We saw already LKDB, the database name. Other environment variables that can be set, e.g. by `source ./activate` or directly in terminal mode by e.g., `export LKDB=$PWD` are
LKDB LKE_DOMAIN LKE_GROUP LK_PATH LK_PUBKEY LK_ENCKEY LK_KEYNAME LK_PRIVATE LK_REPO
4.2 ABE
Linkspace packets have no concept of encoding formats. All fields are fixed length or prefix their exact length. For example, the point hash, group and pubkeys are 32 bytes. These 32 bytes are encoded in URL-safe, no-padding base64. These are string that are practically unreadable as e.g. `j0SWgDvDOLIEny0eEoWgDKWyxpmRmavdvbtXfQYz2bE`. ABE (ASCII Byte Expression) syntax allows to name and handle to long bytes as the long strong above is represented in readable format as [#:pub].
The ABE tools `lk encode ….` and `lk eval ….` are to encode input into ABE or to evaluation an ABE expression. ABE is a small byte templating language included for convenience. The syntax is the same for all programming language. ABE is also used for CLI arguments.
Encoding and evaluation commands are illustrated below:
printf "hello world/nl \n" | lk encode -i printf "emoji ⌨ \n" | lk encode -i
ABE is evaluated by substituting an expressions ( [..] ) with its result. For example in [u8:97], the function 'u8' is called with the arguments ["97"]. The function 'u8' reads the decimal string and writes it as a byte. The byte 97 equals the character 'a'. The byte 99 equals the byte 'c'.
lk eval "ab[u8:99]" | xxd
00000000: [1;32m61[0m[1;32m62[0m [1;32m63[0m [1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m[1;31m [0m [1;32ma[0m[1;32mb[0m[1;32mc[0m
The ABE functions can shorten your code, and almost every CLI argument is an ABE expression.
But knowing all of ABE's features it not required to use linkspace
5 Summary
List of lk commands
The linkspace help command `lk -h` or `lk –help` provide list below. lk scan -h for example explains that `lk scan –bare –private rw -m log-desc` scans a database on all points (bare) including the local group ones (private r) in een scan-mode (m) logical timestamp descending. See the help output for more or `lk format –help`, a special one help output with far more detailed information.
Commands: datapoint point - create a datapoint linkpoint point - create a linkpoint keypoint point - create a keypoint multipoint create multiple points from an arbitrary stream of data see `lk link` to create link to a stream of points * link point/stream - create a point with links to points read from stdin rewrite point/stream - rewrite point read from stdin eval abe - eval ABE expression format abe/stream - eval expression for each point from stdin encode abe - encode input into abe print-query query - print full query from common aliases identity public key - generate / print a signing key commons commons - subcommand with various common formats and protocols build on top of linkspace do stream - for each point run a command filter stream - filter a stream of points tee stream - copy input to multiple destinations edit-xheader stream - mutate the xheader of points init system - create a new instance (alternative to the '--init' argument) save system - save points to the database scan system - scan for matching points in then database watch system - wait for matching points written to the database tap system - combine scan & watch scan-hash system - scan for a point by their hash in the database tap-hash system - scan or watch for point by their hash remove system - remove points matching a query multi-query system - read a stream of queries get-links system - append known link.ptr points db-check system - do a db check sort (experimental) stream - buffer points and write them back in some sorted order match (experimental) stream - patternmatch points and execute for matching case
That's it for this quick introduction.
For a more in-depth technical guide or the library API see the Guide.