Quick start with Python
This tutorial shows how to work with linkspace and Python. It starts with turn a light on and expand with more and more requirements. For each requirement a possible Python program is shown.
Let's start to see if the linkspace Python library is available on this computer. This is done by creating a toggle simulating that a light is turned on.
Turn on a light
The most simple simulation of a light turned on is a simple datapoint with the text content "light on"
from linkspace import *
point = lk_datapoint("light on")
print(point.data_str)
print(point.hash)
light on lQh39e6E8VcxT7xR39_2PR5JDLAXjEjxvRaSmLtAPJw
A datapoint can hold up to maximum 216-512 bytes, around 65.000 bytes.
The hash fills up this much memory: 00000000000000000000000000000000 (32 bytes)
A hash of 32 byte is equal to 256 bit, IPv4 is 32 bit, IPv6 128 bit.
The chances of getting the same hash twice by accident is measured in meaningless improbabilities; such as twice guessing the same atom in the observable universe - at the same moment in time since the big bang. The chances of getting the same hash with the same data are 100%.
Turn on a remote light
Assume that the control program for lights is on one computer and the light itself is controlled by another computer. Below you see how the `toggle` point created above is serialized such that it can be transmitted or stored and at the receiving end, in another program or at another computer or latter reread and deserialized.
from linkspace import *
datapoint_generated = lk_datapoint(data="data in keypoint 1 to be serialized")
print("point with generated pubkey: ", datapoint_generated.hash)
point_bytes = lk_serialize(datapoint_generated)
[receiving_point, nr] = lk_deserialize(point_bytes)
print("a point had been successfully created by a deserialize call:")
print("deserialized point: ", receiving_point.hash)
print("")
try:
print("Changing the data will fail the deserialize")
point_bytes = point_bytes.replace(b"to be serialized", b"to be corrupted ")
[point, nr] = lk_deserialize(point_bytes)
print("print nr: ", nr)
except Exception:
print("An exception occurred: Could not deserialize as one of more bytes changed between serialize and deserialize")
point with generated pubkey: C87npqMTxgjhVzuNTsQ1BFjynt_DGMDGtlGbFHcyA2o a point had been successfully created by a deserialize call: deserialized point: C87npqMTxgjhVzuNTsQ1BFjynt_DGMDGtlGbFHcyA2o Changing the data will fail the deserialize An exception occurred: Could not deserialize as one of more bytes changed between serialize and deserialize
Turn on a specific light
Our messages should indicating which specific light bulb we want to turn on. But before we can do so, one need to understand valid/non-valid text. Linkspace uses bytes as characters. Python print messages and HTTP links uses ASCII characters. Not all bytes are valid ASCII's.
ASCII text versus Linkspace bytes
When you open your browser and use HTTP to visit http://192.168.1.5:8080/garage/above-door,
it opens a stream to 192.168.1.5:8080 and sends a header starting with .
The header field /garage/above-door is the 'path'.
Linkspace does a similar thing with paths like /garage/above-door, but it differs in an important way.
HTTP headers like GET ... HTTP/1.1, are part of the message defined by a protocol.
The HTTP protocol says header must only contain valid text.
Light names with the name: 💡, /, or \0 ( a value of all 0 bits ) are not valid.
A lot of programs as well as HTTP expect valid text only. They break on printing a hash if a field contains e.g. the number \0.
The solution is to use ABE ( ASCII byte expressions ). When ABE has to print bytes it escapes the unprintable ASCII characters with \x00 as well as '\', '/', ':', ' ', and '[]'. ABE is not part of the linkspace protocol, but it is the normative way to interface between bytes as used in Linkspace and e.g. print statement in Python where you need to output on a terminal or screen printable strings.
print( [ byte for byte in point.hash ] ," are 32 bytes ")
print(point.hash, "are 43 letters in base64 encoding " )
print(lka_eval2str("[hash:str]", point=point), "works using the library - but ABE's main usecase is the CLI")
[149, 8, 119, 245, 238, 132, 241, 87, 49, 79, 188, 81, 223, 223, 246, 61, 30, 73, 12, 176, 23, 140, 72, 241, 189, 22, 146, 152, 187, 64, 60, 156] are 32 bytes lQh39e6E8VcxT7xR39_2PR5JDLAXjEjxvRaSmLtAPJw are 43 letters in base64 encoding lQh39e6E8VcxT7xR39_2PR5JDLAXjEjxvRaSmLtAPJw works using the library - but ABE's main usecase is the CLI
Path to a specific light
A datapoint only contains 'hash' and 'data' as well as several empty fields. A linkpoint has the header fields 'path' filled. This path we can use to turn a specific light. The path is a point field that among other things let a device filter & select in Linkspace just the set of points it needs.
point = lk_linkpoint(data="", path=["garage","above-door"]) print(point.path_list) path ="/".join(point.path_str) print(path)
pre rest92 Required60 ucontentsize86 pre rest92 Required60 ucontentsize86 pre rest92 Required60 ucontentsize86 [b'garage', b'above-door'] pre rest92 Required60 ucontentsize86 garage/above-door
Path parts know their length and need no special characters.
That means they it can be garage, but also special characters not usable in ASCII text as \0, 💡, or /.
A second field all linkspoints have is the 'stamp'. Its the current time in microseconds since UNIX epoch. It offers some ordering between points.
In the vast majority of cases you can trust the stamp's validity, especially when it is ordering points signed by one pubkey. But it is neither unique, nor a proof of ordering.
You'll need to consider the impact if a device starts faking its stamps. In the coming chapters we'll show how the `links` fields creates links between points by their hash. Such links are indisputable proof that one point was created before another.
Turn on multiple lights by you
You light up the room! - Sweet, but less so when it is everybody including online strangers.
Next, limit control of the light to only messages coming from you. We need to include security. This requires (public-)key cryptography. Key cryptography works by doing a lot of math on a hash of a message. A hash which supernets have by definition.
Key cryptography in linkspace is used in keypoints. These are signed linkpoints - with all the same fields - and include a public key (i.e. pubkey) and signature.
The pubkey field is a number just like the hash.
To determine if a message was created by you, simply check if the pubkey matches yours.
# create a new identity
your_key = lki_generate()
print("pubkey:", your_key.pubkey)
# saving an (encrypted) key for later
enckey = lki_encrypt(your_key,b"my password")
print("enckey:", enckey)
# opening the enckey
your_key = lki_decrypt(enckey,b"my password")
# Your message
point = lk_keypoint(data="toggle", path=["garage","above-door"], key=your_key )
print("Authorized:", point.pubkey == your_key.pubkey)
# some random message
random = lk_linkpoint(data="toggle", path=["garage","above-door"])
print("Authorized:", random.pubkey == your_key.pubkey)
random_signed = lk_keypoint(data="toggle", path=["garage","above-door"], key=lki_generate())
print("Authorized:", random_signed.pubkey == your_key.pubkey)
pubkey: A5olJLPraqwx5Qu_zCQ13gXWIBsRagrMuSFr2w3gDIA enckey: $lki$argon2d$v=19$m=19456,t=3,p=1$A5olJLPraqwx5Qu_zCQ13gXWIBsRagrMuSFr2w3gDIA$i4WVTY1XKKPKytgV5-WyN9KEYIesfJx8do_u5OyLg-E pre rest188 Required60 ucontentsize188 pre rest188 Required60 ucontentsize188 pre rest188 Required60 ucontentsize188 Authorized: True pre rest92 Required60 ucontentsize92 pre rest92 Required60 ucontentsize92 pre rest92 Required60 ucontentsize92 Authorized: False pre rest188 Required60 ucontentsize188 pre rest188 Required60 ucontentsize188 pre rest188 Required60 ucontentsize188 Authorized: False
The first call, lki_generate, generates a public/private (private of secret) key set. Next you need to encrypt immediately your private key with a password resulting in the encrypted key enckey. As shown below you would do this elsewhere and export the pubkey and enckey. At this moment you have a private/public key. This lets you create keypoints.
# save our previously created keys
open("encrypted-keys.txt", "w").write(enckey+"\n")
open("authorized.txt", "w").write(str(your_key.pubkey)+"\n")
accept = PubKey( open("authorized.txt").read().strip() )
unsigned_linkpoint = lk_linkpoint(path=["this","doesn't","work"])
keypoint = lk_keypoint(key=your_key, path=["garage","door"])
for point in [ unsigned_linkpoint, keypoint]:
if not point.pubkey == accept:
print(f"DENIED: {point.path_str} by {point.pubkey}")
continue
# point.path_list is a list of parts bytes
# point.path_str is a list of parts bytes encoded in ascii-byte format
path = "/".join(point.path_str)
# os.system(f"toggle-light {path}")
print(f"toggle-light {path}")
pre rest92 Required60 ucontentsize86 pre rest92 Required60 ucontentsize86 pre rest180 Required60 ucontentsize176 pre rest180 Required60 ucontentsize176 pre rest92 Required60 ucontentsize86 pre rest92 Required60 ucontentsize86 pre rest92 Required60 ucontentsize86 DENIED: ['this', "doesn't", 'work'] by AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA pre rest180 Required60 ucontentsize176 pre rest180 Required60 ucontentsize176 toggle-light garage/door
Keypoints are proof that the points: data, public key, and other fields, were combined into a unit on a device holding the private key. The machine controlling the light is set up to only react to these kinds of points. A device without that specific private key is unable to create a point that toggles the light. Next we consider what we need to support more people controlling the lights.
Turn on lights by more people
You call, and there is light. For you. For me?
To accept who can use the light, we use the pubkey of those who are authorized.
## Previously: accept = PubKey( open("authorized.txt").read().strip() )
accept_set = { PubKey( line.strip() ) for line in open("authorized.txt").read()}
for point in [ unsigned_linkpoint, keypoint]:
## Previously: if not point.pubkey == accept:
if not point.pubkey in accept_set:
print(f"DENIED: {point.path_str} by {point.pubkey}")
continue
path = "/".join(point.path_str)
print(f"toggle-light {path}")
On the receiving end we can also indicates the intended set of recipients by two more relevant fields: the group and domain.
# A group is 32 bytes, the all zeros is the local or private group # A domain is 16 bytes - left padded with \0 lp = lk_linkpoint(data="toggle", domain="example-💡".encode(), group=[0] * 32) print(lp.group) print(lp.domain.utf8()) # strip padding - read as utf8 print(str(lp.domain)) # with padding - ascii byte encoding
pre rest68 Required60 ucontentsize66 pre rest68 Required60 ucontentsize66 pre rest68 Required60 ucontentsize66 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA pre rest68 Required60 ucontentsize66 example-💡 pre rest68 Required60 ucontentsize66 \0\0\0\0example-\xf0\x9f\x92\xa1
If none is provided the public group is used, indicated with [#:pub]. The local group is [#:0]. The domain separates one app from another. Domains can be thought of as port numbers.
Together with the path they determine a point's "space" as in "domain:group:/path/example" Where any component can be left off to use the default.
The default domain/group is taken from the LKE_DOMAIN / LKE_GROUP environment variables.
Turn on lights by light access
For our lights we'll sketch out a system where an 'admin' must grant permission slips to other people. The computer controlling our lights can demand anyone first sends their permission followed by a toggle request.
admin = lki_generate() request = lk_keypoint(path=["lamp","garage","above-door"], key=key) permission = lk_keypoint(key=admin, path=["permission", request.hash])
incoming_points = [permission,request]
accepting = {}
for point in incoming_points:
if point.path0 == b"permission" and point.pubkey == admin.pubkey:
ptr = point.links[0].ptr
if ptr not in accepting:
accepting[ptr] = True
else if point.path0 == b"lamps":
if ptr.hash in accepting and accepting[ptr] == True:
accepting[ptr] = False # don't allow reuse
path = "/".join(point.path_str[1:])
print(f"toggle-light {path}")
Two issues pop up. First, if the program stops it forgets its accepted requests. Secondly, if a permission arrives before a request it doesn't work.
Both can be solvable by saving points. The first by writing a new point after fulfilling the request, and the second by reading the points we've saved.
Saving points can be as simple as reading/appending to a file.
Eventually, walking through every message front to back becomes slower than keeping an index in a database. Feel free to pick the database that keeps a log for received orders, and an index by hash and (path,pubkey). However, in the next chapter the use of the library that is include in Linkspace is explained.
Turn on lights using a database
To exchange all kind of (digital) information you can create linkpoints and keypoints. These points can be exchanged between programs. Each program can store all points it creates and receives from others. But the fundamental design feature of linkspace is that as each point has a unique number and as a result let's store all points in a shared file/database. We start with two/three (switch/lamp/log) programs that exchange points on one computer, later we'll explain how this can be done between programs running on different computers interconnected by TCP/IP.
The linkspace LkSystem
The linkspace database, or linkspace system LkSystem is created by the lks_open call. It is possible to use an in memory solution (lks_open_inmen(str)), but the standard call opens a file structure. That file/directory structure can be access by the program threads, but also by cli (command line instruction `lk`). This enables you to monitor the linkspace while the program(s) execute. See the quickstart-cli document.
The linkspace LkSystem itself consists of a database shared between threads and processes on one computer. Once a point is created it is stored in the database after calling the lks_save. And per thread there is a set of callbacks called whenever a matching packet is observed. For this to happen, the lks_process call updates the view of the thread on the database to include new points saved from other applications and to trigger the registered callbacks.
Scan/watch/tap and queries on the linkspace database
To scan for points of interest queries are created and with the lks_scan call the LkSystem returns those points that satisfy the query. This is shown in the monitor process. It is just a call that reports the list of all point in the database.
# just a reminder: have you executed 'source ./activate'?
from linkspace import *
lks_open()
def log_report(point):
print(f"At {point.stamp} Garage lamp above door was: {point.data_str}")
query_for_scan = lkq_space("example:[#:pub]:/garage/above-door",":watch above-door")
amount =lks_scan(query_for_scan, on_point=log-report)
print(f"there were {amount} points")
The lamp process monitors when/whether the lamp switch is turned on or off. This is do by a lks_watch. Once an update of the database with a lks_process takes place, LkSystem checks if a certain query condition is fulfilled and results the result by calling the callback function specified in the lks_watch call.
from linkspace import *
lks_open()
query = lkq_add(lkq_space("example:[#:pub]:/garage/above-door",":watch above-door"))
# toggle is the callback function
def toggle(point, _):
print("Garage lamp above door is:", point.data_str)
lks_watch(query, on_point=toggle)
while True:
try:
last_stamp = lks_process_while(watch=b"above-door")
except KeyboardInterrupt:
break
In the example the lks_process_while is used as in the case of this simple program the program/thread had nothing else to do and can just wait till the condition is fulfilled.
For completion also the switch.py program
from time import sleep
from linkspace import *
lks_open()
print(lke_get()) #from work environment you can extract the linkspace database directory in the file structure
while True:
try:
# create a linkpoint to turn-on (1) a specific lamp above the door in the garage
lp = lk_linkpoint(data="1", domain="example",path=["garage","above-door"])
lks_save(lp)
# wait 15 sec to toggle it off again
sleep(15)
lks_save(lk_linkpoint(data="0",domain="example",path=["garage","above-door"]))
sleep(15)
except KeyboardInterrupt:
break
Next to the lks_scan and lks_watch as used above, there is a third function lks_tap. This function is a combination of a scan first followed by a watch.
Summary
Here are some notes on what sets linkspace apart from similar projects.
- Toggling light bulbs requires few dependencies
- Not a framework - you call the library, not the other way around
- A fast hash and security build-in from the start
- Domains - the focus is on being a generic tool, not solve 1 use case
- Groups - not everything is in a public group
- Path - resembles organisation of data in well-known file directories
- Spaces - no fixed dependency on TCP, IP, or any global network