Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

System Architecture

This chapter provides a deeper look at how AutoCore works internally. You don’t need to understand all of this to use AutoCore, but it will help you debug issues and make better design decisions.

Architecture Diagram

┌─────────────────────────────────────────────────────────────────┐
│                        AutoCore Server                           │
│                                                                   │
│  ┌──────────┐ ┌──────────┐ ┌────────────┐ ┌──────────────────┐ │
│  │  System   │ │    GM    │ │  Datastore  │ │   Module IPC     │ │
│  │ Servelet  │ │ Servelet │ │  Servelet   │ │   Server         │ │
│  └─────┬─────┘ └─────┬────┘ └──────┬─────┘ └────────┬─────────┘ │
│        │              │             │                 │           │
│        ▼              ▼             ▼                 ▼           │
│  ┌─────────────────────────────────────────────────────────────┐ │
│  │               Shared Memory (autocore_cyclic)                │ │
│  │  ┌──────────┐  ┌───────────┐  ┌─────────────┐  ┌─────────┐ │ │
│  │  │ Variables │  │  Signals  │  │   Direct    │  │ Events  │ │ │
│  │  │ (I/O)    │  │  (Tick)   │  │   Mapping   │  │ (Sync)  │ │ │
│  │  └──────────┘  └───────────┘  └─────────────┘  └─────────┘ │ │
│  └──────────▲─────────────────────────────▲────────────────────┘ │
│             │   Zero-Copy R/W             │   Zero-Copy R/W      │
│             │   (every cycle)             │   (every cycle)       │
└─────────────┼─────────────────────────────┼──────────────────────┘
              │                             │
   ┌──────────┴──────────┐    ┌─────────────┴──────────────┐
   │   Control Program    │    │      External Modules       │
   │   (your program.rs)  │    │  (EtherCAT, Modbus, etc.)  │
   │                      │    │                              │
   │  autocore-std        │    │  mechutil IPC client         │
   └──────────────────────┘    └──────────────────────────────┘
              │                             │
              ▼                             ▼
   ┌──────────────────────┐    ┌──────────────────────────────┐
   │   Web Console / HMI   │    │    Field Devices              │
   │   (Browser, ws://)    │    │  (Drives, Sensors, I/O)      │
   └──────────────────────┘    └──────────────────────────────┘

Shared Memory Model

Shared memory is the heart of AutoCore’s performance. Instead of sending data through network protocols or message queues, all processes access the same memory region directly.

  1. Allocation: When the server starts, it creates a shared memory segment called autocore_cyclic based on the variables in project.json.
  2. Mapping: The control program and all enabled modules map this segment into their own address space.
  3. Synchronization: The server generates a tick event. The control program waits for this event, reads the memory, processes one cycle, and writes back.

This zero-copy architecture means that I/O data exchange takes nanoseconds, not milliseconds.

The Module System

External modules extend AutoCore’s hardware capabilities. Each module:

  1. Is spawned as a child process by the server on startup
  2. Receives three CLI arguments: --ipc-address, --module-name, and --config
  3. Connects to the server’s IPC port (default 9100)
  4. Receives lifecycle commands: initialize, configure_shm, finalize
  5. Maps shared memory variables to exchange cyclic data
  6. Handles commands routed by the server based on the module’s domain name

Built-in modules:

  • autocore-ethercat: EtherCAT fieldbus master
  • autocore-modbus: Modbus TCP client
  • autocore-labelit: Camera and label inspection

Two spawn paths: supervisor vs. ad-hoc

The server has two places that can launch a module process, and the difference matters:

  • Module supervisor (ModuleSupervisor::start_module). Spawns every module listed in project.json::modules, in order, on server startup. Resolves ${ams.*} placeholders against the AMS registry first; if any are unresolved, the spawn is refused and the module is marked Failed. All three CLI args are passed, including --config <resolved-json>.
  • Ad-hoc (system.load_module). Bootstraps a module that is registered in config.ini::[modules] but NOT declared in project.json. Only two CLI args are passed: --ipc-address and --module-name. No --config — by design, because there’s nothing in project.json to serialise.

A module that’s launched via the ad-hoc path receives no project configuration. That’s the right behaviour for system.load_module’s intended use (bootstrap before the module is in project.json), but it was historically the wrong behaviour as a fallback: if the supervisor refused a project-declared module, an ad-hoc launch would silently substitute a configless process, and <module>.status would report empty arrays with no obvious cause.

Current behaviour: system.load_module refuses any name that exists in project.json::modules and returns an error pointing at the supervisor log and acctl validate. The error reads:

Module ‘X’ is declared in project.json — refusing ad-hoc launch. If the supervisor refused to start it, check the server log for the reason (commonly UnresolvedAmsPlaceholders) and run acctl validate against this project.

Validating before spawn: system.validate_project

The server exposes system.validate_project as a read-only IPC command that runs the same checks the supervisor would apply at spawn time, plus all the AMS-entry integrity checks. acctl wires this into acctl validate, acctl sync (push), and acctl codegen so bad files never reach module-spawn time. See Chapter 12 for the categories of errors it reports.

Request:

{ "topic": "system.validate_project",
  "data": { "project_json": <Value>?, "module": "<name>"? } }
  • project_json (optional): validate this exact content instead of the currently-loaded file. acctl sends the local file here before pushing.
  • module (optional): scope AMS placeholder reporting to /modules/<name>. AMS-entry and cross-module checks still run.

Response:

{ "ok": true,
  "errors": [
    { "category": "ams.placeholder",
      "severity": "warning",
      "path": "/modules/ni/config/tasks/0/channels/8/create_args/scaled_max",
      "message": "no active asset registered at location `tsdr`",
      "extra": { "placeholder": "${ams.by_location.tsdr.sub.my.capacity}" } }
  ] }

Each finding carries a severity of "error" (default, blocks acctl sync push) or "warning" (surfaces in acctl output and the AIS Placeholder Health panel but does not block sync). The top-level ok is true when no error-severity findings are present — warnings alone do not flip it false. ams.placeholder is the only category currently emitted as a warning; everything else (json, module_schema, ams.registry, ams.calibrations, cross-module variable / link checks) stays as error.

Configuration: config.ini

The config.ini file contains machine-specific settings that stay the same across projects. It is located at:

  • Linux: /opt/autocore/config/config.ini
  • Development: specified with --config flag when running the server
[console]
port = 11969                                        # WebSocket port for CLI and web clients
www_root = /srv/autocore/console/dist               # Path to web console static files

[general]
projects_directory = /srv/autocore/projects          # Root directory for all projects
module_base_directory = /opt/autocore/bin/modules    # Directory containing module executables
port = 8080                                          # HTTP port for the web server
autocore_std_directory = /srv/autocore/lib/autocore-std  # Path to the autocore-std library
disable_ads = 1                                      # Disable TwinCAT ADS compatibility
ipc_port = 9100                                      # TCP port for module IPC
project_name = default                               # Project to load on startup

[modules]
modbus = ${general.module_base_directory}/autocore-modbus
ethercat = ${general.module_base_directory}/autocore-ethercat
labelit = ${general.module_base_directory}/autocore-labelit

The [modules] section maps module names to executable paths. This keeps project.json portable — the same project file works on different machines where modules may be installed in different locations.

The CommandMessage Protocol

All communication in AutoCore — between web clients and the server, between the CLI and the server, and between modules and the server — uses the CommandMessage protocol. Understanding this protocol helps you debug communication issues and write effective HMI code.

A CommandMessage is a JSON object with the following fields:

{
  "transaction_id": 101,
  "timecode": 1768960000000,
  "topic": "gm.motor_speed",
  "message_type": 2,
  "data": null,
  "crc": 0,
  "success": false,
  "error_message": ""
}
FieldTypeDescription
transaction_idnumberUnique ID for matching responses to requests. The server echoes this back. For broadcasts, this is 0.
timecodenumberTimestamp in milliseconds since UNIX epoch.
topicstringThe FQDN (Fully Qualified Domain Name) of the resource. The first segment routes to the appropriate module or servelet (e.g., gm, modbus, ethercat, datastore).
message_typenumberThe operation to perform (see table below).
dataanyThe payload. For a Write, this is the value to set. For a Read Response, this is the value retrieved.
crcnumberOptional CRC32 checksum for message integrity verification. Defaults to 0.
successbooleantrue if the operation succeeded, false if it failed. Only meaningful in responses.
error_messagestringHuman-readable error description if success is false. Otherwise empty.

Message Types

NameValueDescription
NoOp0No operation. Used for connection testing / ping.
Response1Reply to a previous request. The transaction_id matches the original.
Read2Request to read the current value of topic.
Write3Request to update the value of topic.
Subscribe4Request to receive updates whenever topic changes.
Unsubscribe5Stop receiving updates for topic.
Broadcast6Unsolicited push from server to client (live variable update).
Heartbeat7Keepalive signal.
Control8System control message (initialize, finalize, configure).
Request10Generic RPC call. The topic implies the action, data contains arguments.

The protocol follows a REST-like pattern: the topic is the resource (like a URL path), and the message_type is the verb (like an HTTP method).

Common Workflows

Reading a variable:

// Request (Client → Server)
{ "transaction_id": 101, "topic": "gm.motor_speed", "message_type": 2, "data": null }

// Response (Server → Client)
{ "transaction_id": 101, "topic": "gm.motor_speed", "message_type": 1, "data": 1500, "success": true }

Writing a variable:

// Request
{ "transaction_id": 102, "topic": "gm.motor_speed_setpoint", "message_type": 3, "data": 1200 }

// Response
{ "transaction_id": 102, "topic": "gm.motor_speed_setpoint", "message_type": 1, "success": true }

Subscribing to live updates:

// Subscribe request
{ "transaction_id": 103, "topic": "gm.motor_speed", "message_type": 4, "data": {} }

// Confirmation
{ "transaction_id": 103, "topic": "gm.motor_speed", "message_type": 1, "success": true }

// Subsequent broadcasts (sent automatically when value changes)
{ "transaction_id": 0, "topic": "gm.motor_speed", "message_type": 6, "data": 1485, "success": true }

FQDN Routing

The topic string determines where a message is routed. The first segment (before the first .) is the domain, which maps to a servelet or module:

DomainRoutes ToExample Topics
gmGlobal Memory serveletgm.motor_speed, gm.cycle_counter
systemSystem serveletsystem.get_domains, system.new_project, system.full_shutdown
datastoreDatastore serveletdatastore.calibration.offset
modbusModbus modulemodbus.vfd_01.speed_setpoint
ethercatEtherCAT moduleethercat.clearpath_0.rxpdo_5.controlword
pythonPython serveletpython.run_script

Glossary

TermDefinition
FQDNFully Qualified Domain Name. A dot-separated hierarchical address for any resource in the system. Example: ethercat.servo_drive.rxpdo_1.controlword
PDOProcess Data Object. The cyclic data image exchanged with fieldbus devices every scan cycle.
SDOService Data Object. A request/response protocol for reading or writing individual configuration parameters from a device. Used for acyclic (on-demand) access.
Cyclic dataData exchanged at a fixed interval (every tick). PDO data from EtherCAT slaves is cyclic. Requires deterministic timing.
Acyclic dataData exchanged on demand or at variable intervals. Modbus register reads, SDO access, and CommandMessage requests are acyclic.
Process imageThe complete set of input and output data for all devices on a fieldbus, updated each scan cycle.
Scan cycleOne complete exchange of process data with all fieldbus devices. At a 1 ms cycle time, there are 1,000 scan cycles per second.
ServeletAn internal module within autocore-server that handles a specific domain of messages (e.g., GM servelet, Datastore servelet, System servelet).