Skip to main content
Version: 5.2.0.1

Deserializers

Concept​

All Adapters reading byte streams need a deserializer to parse the content of the stream and create a structured message from it.

:::info Inbound vs. Outbound Note that the terms Inbound channel and Outbound channel do not refer to the direction in which the data flows, but to the initiator of the call.

  • An outbound channel might read data (e.g., the File Reader) and therefore needs a deserializer.
  • An inbound channel might write data (e.g., the HTTP GET Receiver) and therefore needs a serializer. :::

Example: If you want to read a CSV file using the File Reader, you need a CSV deserializer to transform the content into a structured message (a DOM tree).

Creation​

A deserializer is always created in the context of configuring a Stream channel.

  1. Select a deserializer.
  2. Select an appropriate message type (if necessary).
  3. Click on the button marked with ... to configure the deserializer.

:::tip Auto-selection If no deserializer is selected yet (or only the binary reader is selected) and you select a message type associated with a particular deserializer, that deserializer is automatically selected. :::

Input Filters​

Below the deserializer selection line, you can select a Stream filter by double-clicking the arrow -->. This allows you to build a sequence of filters. To configure a filter, double-click on its name in the sequence.

CSV Reader​

Description​

The CSV reader parses CSV data and creates a structured Message from it.

Configuration​

  • Encoding: Encoding for the CSV stream.
  • Separator character: Separator for the columns.
  • Quote character: Values in the columns are surrounded by this character.
  • Skip lines: How many lines shall be skipped at the beginning (e.g., for titles).
  • Ignore empty fields: Check this box to ignore empty fields.

XML Reader​

Description​

The XML reader parses an XML document and creates a structured Message from it representing the DOM tree.

Configuration​

  • Ignore trailing text: If checked, bytes after the XML file content are ignored.

Excel Reader​

Description​

The Excel reader parses Excel files and creates a structured Message from them.

Configuration​

  • Sheet: Insert the name of the sheet to be used.
  • Skip lines: Lines that shall be skipped (e.g., if there is a header title).
  • Ignore empty cells: Check to ignore empty cells.
  • Ignore empty lines: Check to ignore empty lines.
  • Typed: If checked, Orchestra will look up the type of the cell (Number, Date, etc.) instead of treating everything as text.

Stream Parser​

Description​

The stream parser parses an input stream on the basis of a parser description and creates a structured Message from it.

Configuration​

  • Encoding: The stream parser works on text data; therefore, select an encoding used to transform the input byte stream into characters.
  • Prefer content type encoding: Check to use the default encoding of the message.
  • Parser: The stream parser uses a script written in a declarative language. It describes the structure of the input text and how to transform it to a message. Such scripts are contained in scenario elements of type Parser description.
  • Start: Choose an element of the parser description as the start point.

Binary Reader​

Description​

The binary reader reads the input byte stream and creates a Message containing the byte stream. It does not interpret the byte content in any way.

This is typically used to read binary content like a PDF file, allowing Orchestra to route binary data to target systems. In many cases, the Binary Writer is then used to write the content to the target system.

Configuration​

A binary reader cannot be configured.

EDI Message Reader​

Description​

The EDI message reader parses EDIFACT formatted messages and creates a structured Message from them.

Configuration​

  • EDI-Variant: Choose the EDI variant.
  • Mode:
    • Dynamic: Creates the message structure dynamically.
    • Type: Uses an internal message type for deserialization.
  • Type lookup: Choose the font/type that shall be used.
  • Encoding: (Only in type mode) The EDI fields can be specified here.
  • Separators: Separators can be specified; usually, these are . and ,.

Text Reader​

Description​

The text reader parses text data and creates a structured Message from it.

Configuration​

  • Encoding: Choose the encoding which shall be used.
  • Break text on newline: Check if text shall be divided into elements based on new lines.

Type Based Deserializer​

Description​

The type based deserializer parses data according to the information in a Message type with default type INTERNAL.

:::warning Requirement Because processing depends on a message type, you MUST select a Message type describing the structure of the data to parse. In the dropdown box, you can only select message types with format type INTERNAL. :::

Configuration​

The core configuration is contained within the message type itself. You can invoke the Format Editor by clicking the Edit format button.

Delimiter Properties:

PropertyDescription
LevelAssigns consecutive sets of delimiter parameters to delimited nodes in the hierarchy.
DelimiterAny alphanumeric sign(s), including control characters (e.g., \r, \n).
Optional modeSpecifies how delimiters for optional nodes are handled when nodes are absent/empty.
Terminator modeSpecifies how delimiters are handled for a specific terminator node.
Repeat symbolA delimiter used to delimit repetitive occurrences (e.g., *).

Optional Mode Options:

OptionDescriptionInput BehaviorOutput Behavior
NeverDelimiter not allowed if node is absent.b*d^b^
AllowDelimiter allowed in input if node absent, but not emitted.b**d^b^
CheerDelimiter allowed in input if node absent, and emitted.b**d^b**^
ForceDelimiter must appear in input if node absent, and is emitted.b**d^b**^

Escaping: To include a delimiter inside a string literal, you must escape it using the backslash (\).

Additional Configuration: If you click on the Edit button in the deserializer settings (outside the format editor), you have two additional properties:

  • Encoding: The encoding of the stream.
  • Check cardinality: Check if you want Orchestra to validate element cardinality.

PDF Reader​

Description​

The PDF reader parses PDF data and creates a structured Message from it.

Configuration​

The configuration dialog consists of self-explanatory checkboxes (e.g., regarding text extraction or layout).

JSON Reader​

Description​

The JSON reader parses JSON files and creates a structured Message from them.

General Configuration​

In any mode, you can configure:

  • Encoding: Normally JSON is UTF-8. In almost all cases there is no need to change this.
  • Strict: If checked, the message is checked according to RFC 4627 (e.g., valid JSON structure, no multiple top-level values, correctly quoted strings).

The JSON reader works in three primary modes:

1. Generic Mode​

In generic mode, no schema is used.

  • Namespace: Set a namespace used as the default for the created XML document.
  • Array handling: Defines how JSON arrays are converted to XML.
    • As multiple element: Creates repeated elements with the same tag name.
    • As container: Creates a wrapper element around the items.
  • Recursive element names: If "As container" is selected, this option uses the attribute name as the tag name for items.
  • Create null values: If selected, creates an empty element for null. If unchecked, no XML element is created.
  • Create xsi:type: If selected, attempts to detect types (integer, boolean) and adds xsi:type attributes to the XML.

Invalid XML Tag Names: If a JSON property name contains characters invalid in XML (e.g., spaces), the reader changes the tag name and preserves the original in an original_name attribute.

  • JSON: "given name": "Thomas"
  • XML: <given_name original_name="given name">Thomas</given_name>

2. Native Mode​

Native mode creates a native JSON message object instead of a DOM (XML) tree. It validates that the input is valid JSON but does not transform it.

3. Typed Mode​

Generic mode creates XML based solely on structure. Typed mode uses a definition to control the structure, types, and array handling.

A. Typed Mode using XML Schema​

The transformation is controlled by an XML Schema.

  • Root element: You must enter the name of the element in the schema that describes the JSON document structure.
  • Invalid Names: If a JSON property name is invalid in XML, the Schema element must contain an annotation with emdsorch:jsonname pointing to the real JSON name.
  • Arrays: The schema uses maxOccurs="unbounded" to identify arrays.

:::note Ambiguous Arrays For nested arrays (e.g., an array of arrays), the schema requires a wrapper element (dummy element) to correctly interpret the structure. :::

B. Typed Mode using Message Types​

The transformation is controlled by an Orchestra Message Type. A message type acts like an XML schema with a single global element.

  • Invalid Names: In the Message Type editor, select format JSON to add the real property name to the element definition.
  • Map Handling: In JSON, an object can represent a structure or a Map (key-value pairs where keys are dynamic).
    • If an element definition has cardinality > 1 and at least one non-boolean, non-optional XML attribute, it is treated as a Map.
    • The JSON property name becomes the value of the XML attribute.

Example Map Transformation:

  • JSON: { "200": { "description": "OK" }, "404": { "description": "Not Found" } }
  • Message Type: Element response with attribute code.
  • Resulting XML: 
    <response code="200"><description>OK</description></response>
    <response code="404"><description>Not Found</description></response>

REST Services​

::: note[Context] In the context of REST services, you select the entry "REST Service" instead of a specific message type. The reader automatically uses the message type configured in the REST service's Request/Response definition. :::

QDAS Reader​

Description​

QDAS (Advanced Quality Data Exchange Format / AQDEF) is a file format used to store Quality data from measurement devices. The QDAS reader parses this data into a structured Message.

Configuration​

The QDAS reader requires no configuration.

Message Structure​

The reader creates a message according to a specific structure containing DFD (Part data) and DFX (Characteristics/Values).

info

Most files contain either a DFD element (with a Part) or a DFX element, but at least one must exist.