OmniMark

Usage

OmniMark is used to process data, and convert it from one format to another, using a streaming architecture^[2] that allows it to handle large volumes of content sequentially without having to keep it all in memory. It has a built-in XML parser, and support for XQuery via integration with Sedna native XML database. It also has features to process find rules which implement a similar concept to regular expressions, although the pattern expression syntax is more English-like than the regular expression syntax used in Perl and other languages like the Ruby programming language, both of which are more widely used than OmniMark. OmniMark can also be used for schema transformation tasks in the same way as XSLT, but supports switching between procedural and functional code without the need for any additional constructs to support the procedural elements.

History

OmniMark was originally created in the 1980s by Exoterica, a Canadian software company, as a SGML processing program called XTRAN.^[3] XTRAN was later renamed OmniMark, and Exoterica became OmniMark Technologies. The current owners of OmniMark, Stilo International, have their main offices in the UK but also maintain an office in Canada.^[4]

In 1999, OmniMark president and CEO John McFadden announced that OmniMark 5 would be available free of charge, to better compete with Perl.^[5] OmniMark is no longer distributed under such a model.

Programming Model

OmniMark treats input as a flow that can be scanned once, rather than as a static collection of data that supports random access. Much of an OmniMark program is in the form of condition=>action rule where the condition recognizes a length of data to be acted upon and the action specifies what is to be done with the data. There are two kinds of condition:

• An element rule, which can only be used with structured documents (well-formed XML, valid XML, or SGML), recognizes a complete element: the start tag, the element content, and the end tag. Since the content may hold other elements, element rules can operate in a nested fashion. OmniMark manages the nesting in such a way that the element rules can be defined independently of each other.
• A pattern, which can be used with both structured and unstructured documents, recognizes a length of text. Patterns are used like regular expressions in other languages (python, Perl, awk, ...) but have an English-like syntax that facilitates the creation of complex expressions. When parsing a structured document, patterns can be used on text going into the parser or on text coming out of the parser.

Processing Unstructured Input

Find rules are used to apply patterns to unstructured input. Lengths of text are recognized by a pattern that includes temporary pattern variables to capture any part of the text that will be needed in the output. The action uses those variables to produce the required output:

; Change prices from English format to French format
find "$" digit+ => dollars "." digit{2} => cents
    ; output the price in new format
    output dollars || "," || cents || "$"

If two find rules can recognize the same sequence of text, the first rule will “eat” the sequence and the second rule will never see the text. Input that is not recognized by any find rule does not get “eaten” and passes right through to the output.

Processing Structured Input (XML, SGML)

OmniMark sees input as a flow; a program does not hold input data in memory unless part of the data has been saved in variables. As the input flows by, OmniMark maintains an element stack containing information that can be used to guide the transformation of text via the OmniMark pattern-matching facility. When each start tag is encountered, OmniMark pushes another element description on the stack. The element description includes the element name, the attribute names with the types and values of the attributes, along with other information from the parser (such as whether that element is an EMPTY element). When the corresponding end tag is encountered, the element description is popped from the top of stack. With SGML, some tags may be omitted, but OmniMark acts as if the tags were present and in the right places.

OmniMark Element Stack

                       content
                   <h1>   .   </h1>
             <body>    |  .  |     </body>
    <example>      |   |  .  |    |       </example>
             |     |   |  .  |    |      |
             |     |   |  .  |    |      |
             |     |   |  .  |    |      |
             A     B   C  .  D    E      F
                          X

    X: current position in the input document

    Scan        Available
    Location    Information
    A to F      element example
    B to E      elements example, body
    C to D      elements example, body, h1
    C           beginning of content
         D      end of content

An OmniMark program uses element rules to process XML or SGML documents. An element rule:

• gets control just after the start tag has been parsed, and the element description has been pushed on the element stack. The action for the element rule has access to the descriptions of the current element and all the ancestor elements s back to the document root.
• passes control back to the parser by requesting the parsed content of the element in the special value "%c". The content is usually requested for scanning with pattern matching, rather than for storage in a variable.
• gets control again when the corresponding end tag has been parsed, but before the element description is popped from the element stack. The action for the element rule still has access to the descriptions of the current element and all the ancestor elements back to the document root.

Since elements can be nested, several element rules can be in play at the same time, each with a corresponding element description on the element stack. Element rules are suspended while waiting for the parser to finish parsing their content. Only the rule for the element at top of stack can be active. When end of content is reached for the element at top of stack, the action for the corresponding element rule gets control again. When that action exits, the element description is popped and control is returned to the action for the next lower element on the stack. An element rule might simply output the parsed content (as text) and append a suffix:

element "code"
    output "%c"      ; parse and output element content
    do when parent isnt ("h1" | "h2" | "h3" | "h4" | "h5" | "h6")
        output "%n"  ; append a newline if not in a heading
    done

A program does not need to name all of the document elements if the unnamed elements can be given some kind of generic processing:

element #implied
    do when parent is "head"
        suppress        ; discard child elements
    else
        output "%c"     ; parse and output element content
    done

Pattern Matching on Output from the Parser

The parsed content of each element is made available within an element rule and can be modified by a repeat ... scan block that uses patterns to identify the text to be modified:

element "p"      
    ; Change prices from English format to French format
    repeat scan "%c"    ; parse and scan element content
        match "$" digit+ => dollars "." digit{2} => cents
            ; output the price in new format
            output dollars || "," || cents || "$"
        match (any except "$")+ => text
            ; output non-price sequences without change
            output text
        match "$" => text
            ; output isolated currency symbol without change
            output text
    done

The first pattern that matches a leading part of the text will “eat” that text and the text will not be available to the following patterns even if one of the following parts could match a longer leading part of the text. Any leading part that does not match one of the patterns in a repeat ... scan block will be discarded.

Pattern Matching on Input to the Parser

Translate rules get control just after tags have been separated from text but before the completion of parsing. Each translate rule has a pattern that identifies a length of text to be processed. That length of text will not include any tags, but could be as much as the full length of text between two tags.

One use of translate rules is to make a specific change throughout an entire document:

; Change markup character to entity that represented it in the input
translate "&"
    output "&"

The tags before the current point in the input have already gone through the parser, so the element stack already has a description of the element (or nested elements) that contain the text. Consequently, the information on the element stack can be used to control what is done with the text For example, the operation of a translate can be limited to the character content of one or more elements:

; Change prices from English format to French format
translate "$" digit+ => dollars "." digit{2} => cents
        when element is ("p"|"code")
    ; output the price in new format
    output dollars || "," || cents || "$"

Example code

In some applications, much of a document can be handled by a well-designed generic action, so that only a fraction of the document needs special handling. This can greatly reduce the size and complexity of a program and, in the case of XML documents, can make a program very tolerant of changes in the structure of the input document.

A Simple Program

This is the basic "Hello, World!" program:

 process
    output "Hello World!"

Unstructured Input (Text)

This program outputs all words that begin with a capital letter, one word per line, and discards all other text:

 process
    submit file "myfile.txt"
    ; or submit "ANY Text discard lowercase words"
 
 ; output capitalized word, append a newline
 find (uc letter*)=>temp
    output temp || "%n"

 ; discard all other characters
 find any
    ; no output

Structured Input (XML)

OmniMark can accept well-formed XML, valid XML or SGML as structured input. This program outputs a list of first- and second-level headings from an xhtml file, indenting the second-level headings:

; xhtml-headings.xom
; List first- and second-level headings from xhtml or xhtml5 file
; Second-level headings are indented

process
    ; transform the input document
    ; do xml-parse document   ; parse valid XML
    do xml-parse            ; parse well-formed XML
        scan file "example.html"
        output "%c"     ; parse and output document content
    done

element "head"
    suppress            ; discard child elements

element "h1"
    output "%c"         ; parse and output element content
    output "%n"         ; add a line-end

element "h2"
    output "  "         ; indent 2 spaces
    output "%c"         ; parse and output element content
    output "%n"         ; add a line-end

; handle any element not named in explicit rules above
element #implied
    do when parent is "body"
        ; discard all child elements except those named above
        suppress        ; discard child elements
    else
        ; keep the content of any other element
        output "%c"     ; parse and output element content
    done

; discard character content from element "body" if that element
; has mixed content
translate any+ => X when element is body
    ; no output (do nothing with variable "X")

The element #implied rule picks up any element that is not recognized by one of the other element rules.

Structured Input (SGML)

This program replaces the omitted tags in a simple SGML document and outputs something similar to well-formed XML. The program does not translate SGML empty tags correctly to XML empty tags and it does not handle many of the SGML features that can be used in SGML documents.

Program

; Insert omitted tags in SGML document
;
; This program is simplified, for demonstration only,
; The program does not handle many features of SGML
; A more elaborate program would be required to produce
; well-formed XML from most SGML documents.

process
    do sgml-parse document
        scan file "example.sgml"
        output "%c"     ; parse and output document content
    done

element #implied
    output "<%q"        ; begin start tag

    ; write attributes as name="value" pairs
    repeat over specified attributes as attr
        output " "
            || key of attribute attr
            || "=%"%v(attr)%""
    again

    output ">"          ; terminate start tag

    ; write element content
    output "%c"
    
    ; write end tag if element allows content
    output "</%q>"
        unless content is (empty | conref)

; translate markup characters (in text) back to the entities
; that represented them in the original input

translate "&"
    output "&amp;"

translate "<"
    output "&lt;"

translate ">"
    output "&gt;"

Example Input

<!-- A simple SGML document for input to OmniMark demos -->
<!DOCTYPE example [
  <!ELEMENT example   O - (head, body)>
  <!ELEMENT head      O O (title?)>
  <!ELEMENT title     - - (#PCDATA)>
  <!ELEMENT body      - O ((empty|p)*)>
  <!ELEMENT empty     - O EMPTY>
  <!ELEMENT p         - O (#PCDATA)>
  <!ATTLIST P       id    ID    #IMPLIED>
  <!ENTITY  amp     "&">
  <!ENTITY  lt      "<">
  <!ENTITY  gt      ">">
]>
<example>
<title>Title</title>
<body>
<p>Text
<empty>
<p id="P-2">&lt;&amp;&gt;
</example>

Example Output

<EXAMPLE><HEAD><TITLE>Title</TITLE></HEAD><BODY><P>Text</P><EMPTY><P ID="P-2">&lt;&amp;&gt;</P></BODY></EXAMPLE>

Further reading

• Baker, Mark (2000). Internet Programming with OmniMark. Boston: Kluwer Academic Publishers.
• Smith, Norman E. (1998). Practical Guide to SGML/XML Filters. Plano, TX: WordWare Publishing.

References

External links

• Stilo Omnimark
• OmniMark Developer Resources
• OmniMark Programming Principles - General Index (Internet archive snapshot of online book by Errol Chopping)

This computer-programming-related article is a stub. You can help Wikipedia by expanding it.