jakob.scheid
b1ac351ad1
geändert: client/client.py gelöscht: lib/__pycache__/crypto_utils.cpython-313.pyc gelöscht: lib/__pycache__/jebp_utils.cpython-313.pyc gelöscht: lib/__pycache__/terminal_table.cpython-313.pyc geändert: lib/jebp_utils.py gelöscht: server/clients_management/chclient.py gelöscht: server/clients_management/rmclient.py neue Datei: server/data/conf/client_admin_rights umbenannt: server/config/clients/fingerprints -> server/data/conf/client_fingerprints neue Datei: server/data/conf/topics geändert: server/server.py neue Datei: server/utils/clients_management/chclient.py neue Datei: server/utils/clients_management/lsclients.py umbenannt: server/clients_management/mkclient.py -> server/utils/clients_management/mkclient.py neue Datei: server/utils/clients_management/rmclient.py umbenannt: server/clients_management/lsclients.py -> server/utils/topics_management/lstopics.py
167 lines
5.5 KiB
Markdown
167 lines
5.5 KiB
Markdown
# jeb
|
|
|
|
jCloud Event Bus
|
|
|
|
## jebp (jCloud Event Bus Protocol)
|
|
|
|
### Messages
|
|
All messages begin with `0x01`, followed by the content length length. The content length length is the length of the content length and representated by one byte. Then follows the content length and after that the message content.
|
|
|
|
### Handshake
|
|
1. Server: Server protocol, version
|
|
2. Encryption
|
|
1. Client: Serialized client public key
|
|
2. Server: Serialized server public key
|
|
3. Client: Client nonce
|
|
4. Server: Server nonce
|
|
5. Client: Random bytes (encrypted)
|
|
6. Server: received bytes from client (not encrypted)
|
|
|
|
The client closes the connection if the received bytes from server (7.) do not match the generated bytes (6.) to prevent malfunction and test the encryption.
|
|
3. Server authentication
|
|
1. Server: Server certificate
|
|
|
|
The client closes the connection if it does not trust the server certificate.
|
|
4. Client authentication
|
|
1. Client: Client certificate
|
|
|
|
The server closes the connection if the client is unauthorized.
|
|
|
|
|
|
### Commands
|
|
#### `0x11`: Create topic
|
|
##### Request
|
|
`0x11<TOPIC_NAME>`, parameters:
|
|
- `<TOPIC_NAME>`: The name of the topic
|
|
##### Response
|
|
###### Status code
|
|
- `0xa1`: Topic successfully created
|
|
- `0xb4`: Unknown error creating the topic
|
|
- `0xb6`: Topic exists
|
|
- `0xb7`: No administrative rights
|
|
###### Content
|
|
Empty
|
|
|
|
Requires administrative rights.
|
|
|
|
|
|
#### `0x12`: Create record
|
|
##### Request
|
|
`0x12<TOPIC_NAME><CHECKSUM><TIMESTAMP><CONTENT>`, parameters:
|
|
- `<TOPIC_NAME>`: The name of the topic
|
|
- `<CHECKSUM>`: The CRC
|
|
- `<TIMESTAMP>`: The timestamp of the record (the milliseconds since the epoch, formatted as `int64`). If it is `0`, the timestamp the record was is at will be saved.
|
|
- `<CONTENT>`: The record content
|
|
##### Response
|
|
###### Status code
|
|
- `0xa1`: Record successfully created
|
|
- `0xb3`: Unknown error creating the record. Probably the CRC is invalid if the repetition byte is `0x31`
|
|
###### Content
|
|
Empty
|
|
|
|
|
|
#### `0x21`: Subscribe
|
|
##### Request
|
|
`0x21<TOPICS>`, parameters:
|
|
- `<TOPICS>`: The topics, comma-separated
|
|
##### Response
|
|
###### Status code
|
|
- `0xa2`: Successfully subscribed
|
|
- `0xb0`: Unknown error subscribing
|
|
- `0xb1`: At least one of the topics does not exist
|
|
###### Content
|
|
- Status code `0xa0`: Empty
|
|
- Status code `0xb1`: `<TOPICS>`, `<TOPICS>`: The topics that do not exist
|
|
|
|
|
|
#### `0x22`: Fetch records
|
|
##### Request
|
|
`0x22<START_TYPE><START><MAX_BYTES>`
|
|
- `<START_TYPE>`: One byte, if `0xc0`, the `<START>` is the timestamp of the first fetched event, if `0xc1` the offset.
|
|
- `<START>`: The first event timestamp or offset
|
|
- `<MAX_BYTES>`: Maximum bytes sent
|
|
##### Response
|
|
###### Status code
|
|
- `0xa0`: Success
|
|
- `0xb1`: The offset `<OFFSET>` does not exist
|
|
|
|
#### `0x31`: Remove topic
|
|
##### Request
|
|
`0x31<TOPIC_NAME>`, parameters:
|
|
- `<TOPIC_NAME>`: The name of the topic
|
|
##### Response
|
|
###### Status code
|
|
- `0xa2`: Topic successfully removed
|
|
- `0xb8`: Unknown deletion error
|
|
- `0xb9`: Unknown deletion error. The client should attempt to perform the operation again.
|
|
###### Content
|
|
Empty
|
|
|
|
Requires administrative rights.
|
|
|
|
|
|
### Status codes
|
|
The status code consists of two bytes.
|
|
|
|
1\. Byte (status byte): The status
|
|
|
|
2\. Byte (repetition byte): If the operation was unsuccessful, `0x31` if the client should try to perform the operation again, otherwise or if the operation was successful, `0x30`.
|
|
|
|
In this documentation, the *status code* always means the status byte, unless otherwise stated.
|
|
|
|
#### `0xa…`: Success
|
|
- `0xa0`: Reading successful
|
|
- `0xa1`: Creation successful
|
|
- `0xa2`: Deletion successful
|
|
#### `0xb…`: Error
|
|
- `0xb0`: Reading error
|
|
- `0xb1`: Reading error, object does not exist
|
|
- `0xb2`: Reading error, no permission
|
|
- `0xb3`: Creation error
|
|
- `0xb4`: Creation error, object already exists
|
|
- `0xb5`: Creation error, no permission
|
|
- `0xb6`: Deletion error
|
|
- `0xb7`: Deletion error, object does not exist
|
|
- `0xb8`: Deletion error, no permission
|
|
|
|
|
|
|
|
|
|
## Data storage
|
|
The topics consist of partitions and every partition consists of segments. A segment consists of three files,
|
|
- `<BASE_OFFSET>.log`
|
|
- `<BASE_OFFSET>.index`
|
|
- `<BASE_OFFSET>.timeindex`
|
|
|
|
`<BASE_OFFSET>`: The offset of the first record in the segment
|
|
|
|
In the `<BASE_OFFSET>.log` file, the records are saved.
|
|
|
|
In the `<BASE_OFFSET>.index` file, the positions (bytes) of every 1024th record are saved.
|
|
|
|
In the `<BASE_OFFSET>.timeindex` file, the timestampts of every 1024th record are saved.
|
|
|
|
### Data format
|
|
#### `<BASE_OFFSET>.log`
|
|
##### Record
|
|
| Field | Type | Length (bytes) | Description |
|
|
| - | - | - | - |
|
|
| **Offset** | `int64` | 8 | Offset |
|
|
| **Record Length** | `int32` | 4 | Length |
|
|
| **CRC** | `int32` | 4 | Checksum |
|
|
| **Attributes** | `byte` | 1 | Flags, e. g. Compression |
|
|
| **Timestamp** | `int64` | 8 | Milliseconds since Epoch |
|
|
| **Key Length** | `int32` | 4 | The length of the key |
|
|
| **Key** | `bytes` | -- | optional |
|
|
| **Payload length** | `int32` | 4 | The length of the payload |
|
|
| **Payload** | `bytes` | -- | Record payload |
|
|
| **Headers Count** | `int8` | 1 | Number of header pairs |
|
|
| **Header Key / Value** | `int32` | -- | Header pairs |
|
|
|
|
##### Header pairs
|
|
| Field | Type | Length (bytes) | Description |
|
|
| - | - | - | - |
|
|
| Header key length | `int8` | 4 | Length of the header name (key) |
|
|
| Header key | `bytes` | -- | Header name (key) |
|
|
| Header value length | `int8` | 4 | Length of the header value |
|
|
| Header value | `bytes` | -- | Header value | |