Codex Workflow

You can write Codex workflows to provide custom post-processing logic to transform the incoming metric, events, logs, and traces (MELT) data from one form to another. Post-processing of data indicates that no further mutation is done to the incoming data and only new data can be derived during the post-processing stages.

The Codex workflow schema is based on the Serverless workflow specification version 0.8. The schema is written in the JSONata format.

The Codex workflow schema does not support all the constructs available in the Serverless workflow schema. This section describes the constructs of the Serverless workflow schema that are supported in the Codex workflow, along with their variation implemented in the workflow.

Supported Serverless Workflow Constructs

The following Serverless Workflow constructs and parameters are supported in the FSO Codex workflow:

• CloudEvents: It describes the event data in common formats to provide interoperability across services, platforms and systems. The events are either consumed or produced during workflow execution. For more information, see CloudEvents for Codex.

• Function definitions: The reusable functions to declare and invoke services, and to evaluate expressions.

• State definitions: The building blocks of the workflow control logic. It can call the event definitions and function definitions.

• Action definitions: The action invokes services or other workflows in a workflow execution.

• Start definition: It defines the starting state of the workflow. It also defines when or how to create workflow instances.

• End definitions: It defines the completion of a workflow instance execution or a workflow execution path.

• Produced event definition: It defines the event that is produced when a workflow execution completes or during a workflow transitions.

• OnEvent definition: It defines the actions that are to be performed for the event definitions. The event definitions are specified in the eventRefs array.

• FunctionRef definition: It defines the name of the reference functions.

• Transition definition: It defines the name of the next state of the transition in a workflow.

• Event data filters: It filters the event payloads that are consumed in a workflow.

• Action data filters: It filters the state data and the action data results.

• ID: The unique identifier of the workflow.

• Version: The version of the Codex workflow.

• Spec Version: The release version of the Serverless Workflow specification. The spec version is 0.8.

• Name: The name of the workflow.

• Description: The description of the workflow.

• Metadata: The metadata information that you want to add.

For more information about the construct definitions, see the Serverless workflow language specification.

Note: If you use any other construct that is not mentioned in this section, the validation of Codex workflow fails.

CloudEvents

In <your-solution-directory>/objects/codex/<workflow-name>.json file, you can define events. The events can be produced or consumed in a Serverless workflow execution. For more information, see CloudEvents for Codex.

The following parameters are supported for an event definition:

• name (mandatory)

• source (optional)

• type (mandatory)

• kind (optional)

Function Definition

Define the functions in <your-solution-directory>/objects/codex/<workflow-name>.json file. The functions are used to select or manipulate workflow, state data, or both.

The supported parameters of the function definition are:

• name (mandatory)

• operation (mandatory)

• type (optional)

Codex workflow supports the following function types:

• Expression: The following construct is an example of the type expression function. These expressions are written in the JSONata language.

  json

  Copy"functions": [
    {
      "name": "GenerateLogDensityEvents",
      "type": "expression",
      "operation": "metric.distribution.dataPoints.{"attributes": $merge([attributes, {"workflow": "metricToEvent" }, {"logDensity" : (distributionValue.intSum > 20 ? "HIGH":"LOW")}]), "entities": $$.metric.entities, "timestamp": timestamp, "type": "logs:logdensity", "spanId": "id", "traceId": "tid" }"
    }
  ]
  

• Custom: The following construct is an example of the type custom function.

Copy"functions": [
    {
      "name": "createEntity",
      "type": "custom",
      "operation": "createEntity"
    }
  ],

State Definition

Define a state to control the flow logic of the Codex workflow execution in <your-solution-directory>/objects/codex/<workflow-name>.json file.

The supported Serverless workflow states are:

• Event state

• Operation state

• Switch state

• Parallel state

Event State

It is used to decide which type of MELT data is ingested to the Workflow. The event state must be used as the starting state. Hence, all Codex Workflows start with this state.

The supported properties of an event state are:

• name (mandatory)

• type (mandatory)

• end (mandatory if transition is not defined)

• stateDataFilter (optional)

• transition (optional)

• onEvents (mandatory)

The following example shows an event state:

Copy{
      "name": "EventReceived",
      "type": "event",
      "onEvents": [
        {
          "eventRefs": [
            "EventEnriched"
          ],
          "eventDataFilter": {
            "data": "${ data }"
          }
        }
      ],
      "transition": "EventFilter"
}

Operation State

This state enables you to provide actions to the workflows. The actions can contain functions that perform meaningful state transformation.

The supported properties of an operation state are:

• name (mandatory)

• type (mandatory)

• end (optional)

• stateDataFilter (optional)

• transition (mandatory if end is not defined)

• actionMode (optional)

• actions (mandatory)

The following example shows an operation state:

Copy{
      "name": "GenerateMeasurement",
      "type": "operation",
      "actions": [
        {
          "functionRef": {
            "refName": "CreateMeasurementFunction"
          },
          "actionDataFilter": {
            "toStateData": "${ measurement }"
          }
        }
      ],
      "end": {
        "terminate": true,
        "produceEvents": [
          {
            "eventRef": "RawMeasurementCreated",
            "data": "${ measurement }"
          }
        ]
      }
    }

Switch State

This state enables you to target a subset of the incoming MELT data and filter out the non-matching data.

The supported properties of a switch state are:

• name (mandatory)

• type (mandatory)

• end (optional)

• stateDataFilter (optional)

• dataConditions (mandatory)

• defaultCondition (mandatory)

The following example shows a switch state:

Copy
{
      "name": "EventFilter",
      "type": "switch",
      "dataConditions": [
        {
          "condition": "type = \"spacefleet:newsfeed\"",
          "transition": "GenerateMeasurement"
        }
      ],
      "defaultCondition": {
        "end": true
      }
    }

Parallel State

This state defines a collection of branches that are executed in parallel. A parallel state can be seen a state that splits up the current workflow instance execution path into multiple ones, one for each branch. These execution paths are performed in parallel and are joined back into the current execution path.

The supported properties of a parallel state are:

• name (mandatory)

• type (mandatory)

• branches (mandatory)

• name (mandatory)

• actions (mandatory)

• timeouts (optional)

• stateDataFilter (optional)

• transition (mutually exclusive with end)

• end (optional)

The following example shows a parallel state:

Copy{
      "name": "CreateEvent",
      "type": "parallel",
      "timeouts": {
        "stateExecTimeout": "PT1S"
      },
      "branches": [
        {
          "name": "Initialize Event",
          "actions": [
            {
              "functionRef": {
                "refName": "InitializeEventFunction"
              }
            }
          ]
        },
        {
          "name": "Set Attributes",
          "actions": [
            {
              "functionRef": {
                "refName": "SetAttributeFunction"
              }
            }
          ]
        },
        {
          "name": "Add Additional Attributes",
          "actions": [
            {
              "functionRef": {
                "refName": "AdditionalAttributeFunction",
                "arguments": {}
              }
            }
          ]
        }
     ],
      "stateDataFilter": {
        "input": "${ input }",
        "output": "${ { \"output\": event } }"
      },
      "end": {
        "terminate": true,
        "produceEvents": [
          {
            "eventRef": "EventCreated",
            "data": "${ output }"
          }
        ]
      }
    }

Common Constructs

In addition to the event definition, function definition, and state definition constructs, you can use the The following common constructs and their parameters in your Codex workflow. For more information, see the Serverless workflow language specification.

Workflow Validation Checklist

In addition to the preceding constructs and parameters, ensure the following conditions are met in a Codex workflow:

• All events defined in the workflow are CloudEvents objects.
• In Operation state, the ActionMode is always SEQUENTIAL.
• The state you specify in the Start field is available in your workflow.
• In State definition, the type must be one of the following:
  • event
  • operation
  • switch
  • parallel
• You cannot define circular dependencies across states. For example, the subsequent states cannot point back to the previous states in a workflow.
• You must not define duplicate states, events, or functions.
• All kinds of reference points to the objects must exist in the workflow. For example, you must pre-define any references to the event definitions and the function definitions inside a state, in the workflow.
• The ExpressionLang field in the workflow must have the value JSONata. The field value is case-insensitive. If the field value is not defined, JSONata is considered by default.