Isn't AXON as readable as YAML?

This post is dedicated to comparing of readability of AXON and YAML for configuration.

Read more…

What is recordclass

Recordclass is a library for creation of record-like classes – "mutable" analog of collections.namedtuple. They support same API as namedtuples. In addition records support assignment operations.

Read more…

AXON 0.6

There is new 0.6 release of pyaxon — python library for AXON.

Read more…

AXON: The best of JSON, XML and YAML

This post continue the series about AXON and pyaxon.

AXON combines the best of JSON, XML and YAML. Let's explain that using pyaxon – reference python implementation of AXON.

Read more…

AXON object serialization example

This post continue series about AXON and pyaxon. Now we consider some examples of object serialization/deserialization using AXON.

Read more…

AXON is a "better" version of YAML

This post continue the series about AXON and pyaxon.

YAML is one of popular serialization formats. It combines orthogonal features that are suitable both for markup and serialization by design.

AXON supports a sort of YAML-like formatting without of braces only for named complex values. AXON text in this kind of formatting is easier for reading and writing by humans.

Read more…

AXON overcomes limitations of JSON

This post continue the series about AXON and pyaxon.

JSON has several limitations:

  • keys are always enclosed in " even if they are identifiers;
  • there are no comments;
  • there are no syntactic support of decimal and date/time values;
  • there is no support for stream of values;

AXON is designed in order to overcome much of them.

Read more…

AXON as fat-free XML

In this post, we will consider one advantage of AXON (see also early posts). It allows to resolve problems that arise when someone will try to translate XML to JSON. The root of this problem is in incompatibility of data models of XML and JSON. XML represents attributed trees with tagged nodes, but JSON represents compositions of arrays and associative arrays. This makes translation of XML difficult. You have to translate XML to fatty JSON (with conventions) in order to save initial structure of XML or have to reorganize initial structure in order to produce more optimal JSON. In the last case, inverse transformation is impossible without of losses. AXON exactly allows to represent XML document without losses.

Read more…

AXON with Python

AXON is eXtended Object Notation. It's a simple notation of objects, documents and data. It's also a text based serialization format in first place. It tries to combine the best of JSON, XML and YAML.

pyaxon is reference implementation of the library for processing AXON with python.

In [2]:
from __future__ import print_function, unicode_literals
from axon import loads, dumps
from pprint import pprint

There are two API functions loads/dumps for loading/dumping from/to unicode string.

By default loading and dumping are safe. By the word "safe" we mean that there is no user code is executed while loading and dumping. Unicode strings are converted only into python objects of given types. There is "unsafe" mode too. It allows to transform unicode string into user defined objects and to dump objects into unicode string under user control. But this is the topic of another post.

Simple example

In [3]:
text = '''\
note {
   from: "Pooh"
   to: "Bee"
   posted: 2006-08-15T17:30
   heading: "Honey"
   "Don't forget to get me honey!" }
'''
vals = loads(text)

Here vals is always list of objects.

In [4]:
ob = vals[0]
print(type(ob))
print(ob)
<class 'axon._objects.Node'>
note{from: 'Pooh', to: 'Bee', posted: datetime.datetime(2006, 8, 15, 17, 30), heading: 'Honey' "Don't forget to get me honey!"}

We see that the message is converted to the instance of class Node. Attribute vals.sequence is the sequence containing attributes and values:

In [5]:
print(type(ob.__vals__))
print(ob.__vals__)
<class 'list'>
["Don't forget to get me honey!"]
In [9]:
print(ob[0])
Don't forget to get me honey!

For dumping objects there are three modes. First mode is compact:

In [7]:
print(dumps([ob]))
note{from:"Pooh" to:"Bee" posted:2006-08-15T17:30 heading:"Honey" "Don't forget to get me honey!"}

Second mode is pretty dumping mode with indentations and without braces:

In [8]:
print(dumps([ob], pretty=1))
note
  from: "Pooh"
  to: "Bee"
  posted: 2006-08-15T17:30
  heading: "Honey"
  "Don't forget to get me honey!"

Third mode is pretty dumping mode with indentation and braces:

In [10]:
print(dumps([ob], pretty=1, braces=1))
note {
  from: "Pooh"
  to: "Bee"
  posted: 2006-08-15T17:30
  heading: "Honey"
  "Don't forget to get me honey!"}

At the end let's consider JSON-like representation too:

In [11]:
text = """\
{note: {
    from: "Pooh"
    to: "Bee"
    posted: 2006-08-15T17:30
    heading: "Honey"
    body: "Don't forget to get me honey!"
}}
"""
vals = loads(text)

It has converted into python dicts:

In [12]:
pprint(vals)
[{'note': {'body': "Don't forget to get me honey!",
           'from': 'Pooh',
           'heading': 'Honey',
           'posted': datetime.datetime(2006, 8, 15, 17, 30),
           'to': 'Bee'}}]

Compact dump is pretty small in size.

In [13]:
print(dumps(vals))
{note:{body:"Don't forget to get me honey!" from:"Pooh" heading:"Honey" posted:2006-08-15T17:30 to:"Bee"}}

Dumping in pretty mode is also pretty formatted.

In [14]:
print(dumps(vals, pretty=1))
{note: {
    body: "Don't forget to get me honey!"
    from: "Pooh"
    heading: "Honey"
    posted: 2006-08-15T17:30
    to: "Bee"}}

JSON-like objects are pretty dumps only in indented form with braces.

JSON-like example

Let's consider now JSON-like example with crossreferences and datetimes:

In [15]:
text = '''\
{
  topic: [
     &1 {python: "Python related"}
     &2 {axon: "AXON related"}
     &3 {json: "JSON related"}
  ]
  posts: [
      { id: 1
        topic: *1
        date: 2012-01-02T12:15+03 
        body:"..." }
      { id: 2
        topic: *2
        date: 2012-01-12T09:25+03 
        body:"..." }
      { id: 3
        topic: *3
        date: 2012-02-08T10:35+03 
        body:"..." }
  ]
}
'''
vals = loads(text)
pprint(vals)
[{'posts': [{'body': '...',
             'date': datetime.datetime(2012, 1, 2, 12, 15, tzinfo=datetime.timezone(datetime.timedelta(0, 10800))),
             'id': 1,
             'topic': {'python': 'Python related'}},
            {'body': '...',
             'date': datetime.datetime(2012, 1, 12, 9, 25, tzinfo=datetime.timezone(datetime.timedelta(0, 10800))),
             'id': 2,
             'topic': {'axon': 'AXON related'}},
            {'body': '...',
             'date': datetime.datetime(2012, 2, 8, 10, 35, tzinfo=datetime.timezone(datetime.timedelta(0, 10800))),
             'id': 3,
             'topic': {'json': 'JSON related'}}],
  'topic': [{'python': 'Python related'},
            {'axon': 'AXON related'},
            {'json': 'JSON related'}]}]

It's easy to see that crossreference links just works:

In [14]:
assert vals[0]['topic'][0] is vals[0]['posts'][0]['topic']
assert vals[0]['topic'][1] is vals[0]['posts'][1]['topic']
assert vals[0]['topic'][2] is vals[0]['posts'][2]['topic']

Pretty dump looks like this one:

In [15]:
print(dumps(vals, pretty=1, crossref=1, sorted=0))
{ topic: [
    &2 {python: "Python related"}
    &3 {axon: "AXON related"}
    &1 {json: "JSON related"}]
  posts: [
    { topic: *2
      date: 2012-01-02T12:15+03
      body: "..."
      id: 1}
    { topic: *3
      date: 2012-01-12T09:25+03
      body: "..."
      id: 2}
    { topic: *1
      date: 2012-02-08T10:35+03
      body: "..."
      id: 3}]}

Note that sorted parameter defines whether to sort keys in dict.

XML-like example

In [16]:
text = '''\
html {
   xmlns:"http://www.w3.org/1999/xhtml"
   head {
     title {"Form Example"}
     link {
        rel:"stylesheet"
        href: "formstyle.css"
        type: "text/css" }}
   body {
      h1 {"Form Example"}
      form { 
         action: "sample.py"
         div {
             class: "formin" 
             "(a)"
             input {type:"text" name:"text1" value:"A textbox"}}
         div {
             class: "formin"
             "(b)"
             input {type:"text" size:6 maxlength:10 name:"text2"}}
         div {
             class: "formb"
             "(c)"
             input {type:"submit" value:"Go!"}}
      }
   }
}
'''
vals = loads(text)
val = vals[0]
print(val)
html{xmlns: 'http://www.w3.org/1999/xhtml' head{title{'Form Example'}, link{rel: 'stylesheet', href: 'formstyle.css', type: 'text/css'}}, body{h1{'Form Example'}, form{action: 'sample.py' div{class: 'formin' '(a)', input{type: 'text', name: 'text1', value: 'A textbox'}}, div{class: 'formin' '(b)', input{type: 'text', size: 6, maxlength: 10, name: 'text2'}}, div{class: 'formb' '(c)', input{type: 'submit', value: 'Go!'}}}}}

Let's examine the value:

In [17]:
print(type(val))
print(val.__vals__)
xmlns = val.xmlns
head, body = val
<class 'axon._objects.Node'>
[head{title{'Form Example'}, link{rel: 'stylesheet', href: 'formstyle.css', type: 'text/css'}}, body{h1{'Form Example'}, form{action: 'sample.py' div{class: 'formin' '(a)', input{type: 'text', name: 'text1', value: 'A textbox'}}, div{class: 'formin' '(b)', input{type: 'text', size: 6, maxlength: 10, name: 'text2'}}, div{class: 'formb' '(c)', input{type: 'submit', value: 'Go!'}}}}]
In [18]:
print(type(head))
title, link = head
print(title)
print(link)
<class 'axon._objects.Node'>
title{'Form Example'}
link{rel: 'stylesheet', href: 'formstyle.css', type: 'text/css'}
In [19]:
h1, form = body
print(type(h1))
print(h1)
<class 'axon._objects.Node'>
h1{'Form Example'}
In [20]:
print(type(form))
print(form.__vals__)
action = form.action
div1, div2, div3 = form
<class 'axon._objects.Node'>
[div{class: 'formin' '(a)', input{type: 'text', name: 'text1', value: 'A textbox'}}, div{class: 'formin' '(b)', input{type: 'text', size: 6, maxlength: 10, name: 'text2'}}, div{class: 'formb' '(c)', input{type: 'submit', value: 'Go!'}}]
In [21]:
print(type(div1))
print(div1.__vals__)
cls1 = getattr(div1, "class")
label1, input1 = div1
print(cls1, label1)
print(input1)
<class 'axon._objects.Node'>
['(a)', input{type: 'text', name: 'text1', value: 'A textbox'}]
formin (a)
input{type: 'text', name: 'text1', value: 'A textbox'}
In [22]:
print(type(div2))
print(div2.__vals__)
cls2 = getattr(div2, "class")
label2, input2 = div2
print(label2)
print(input2)
<class 'axon._objects.Node'>
['(b)', input{type: 'text', size: 6, maxlength: 10, name: 'text2'}]
(b)
input{type: 'text', size: 6, maxlength: 10, name: 'text2'}
In [23]:
print(type(div3))
print(div3.__vals__)
cls3 = getattr(div3, "class")
label3, input3 = div3
print(label3)
print(input3)
<class 'axon._objects.Node'>
['(c)', input{type: 'submit', value: 'Go!'}]
(c)
input{type: 'submit', value: 'Go!'}
In [24]:
print(dumps([val], pretty=1))
html
  xmlns: "http://www.w3.org/1999/xhtml"
  head
    title
      "Form Example"
    link
      rel: "stylesheet"
      href: "formstyle.css"
      type: "text/css"
  body
    h1
      "Form Example"
    form
      action: "sample.py"
      div
        class: "formin"
        "(a)"
        input
          type: "text"
          name: "text1"
          value: "A textbox"
      div
        class: "formin"
        "(b)"
        input
          type: "text"
          size: 6
          maxlength: 10
          name: "text2"
      div
        class: "formb"
        "(c)"
        input
          type: "submit"
          value: "Go!"
In [25]:
print(dumps([val], pretty=1, braces=1))
html {
  xmlns: "http://www.w3.org/1999/xhtml"
  head {
    title {"Form Example"}
    link {
      rel: "stylesheet"
      href: "formstyle.css"
      type: "text/css"}}
  body {
    h1 {"Form Example"}
    form {
      action: "sample.py"
      div {
        class: "formin"
        "(a)"
        input {
          type: "text"
          name: "text1"
          value: "A textbox"}}
      div {
        class: "formin"
        "(b)"
        input {
          type: "text"
          size: 6
          maxlength: 10
          name: "text2"}}
      div {
        class: "formb"
        "(c)"
        input {
          type: "submit"
          value: "Go!"}}}}}

Dataset example

Let's consider simple tabular dataset:

In [26]:
text = '''\
dataset {
   ("id" "date" "time" "territory_id" "A" "B" "C")
   (1 2012-01-10 12:35 17 3.14 22 33500)
   (2 2012-01-11 13:05 27 1.25 32 11500)
   (3 2012-01-12 10:45 -17 -2.26 -12 44700)
}
'''
ob = loads(text)[0]
print(ob.__tag__)
pprint(ob.__vals__, width=132)
print("\nPretty form of dataset:")
print(dumps([ob], pretty=1, hsize=10))
dataset
[('id', 'date', 'time', 'territory_id', 'A', 'B', 'C'),
 (1, datetime.date(2012, 1, 10), datetime.time(12, 35), 17, 3.14, 22, 33500),
 (2, datetime.date(2012, 1, 11), datetime.time(13, 5), 27, 1.25, 32, 11500),
 (3, datetime.date(2012, 1, 12), datetime.time(10, 45), -17, -2.26, -12, 44700)]

Pretty form of dataset:
dataset
  ("id" "date" "time" "territory_id" "A" "B" "C")
  (1 2012-01-10 12:35 17 3.14 22 33500)
  (2 2012-01-11 13:05 27 1.25 32 11500)
  (3 2012-01-12 10:45 -17 -2.26 -12 44700)
In [27]:
from collections import namedtuple
Datarow = namedtuple("Datarow", ob[0])
rows = []
for line in ob[1:]:
    print(line)
    rows.append(Datarow(*line))
for row in rows:
    print(row)
(1, datetime.date(2012, 1, 10), datetime.time(12, 35), 17, 3.14, 22, 33500)
(2, datetime.date(2012, 1, 11), datetime.time(13, 5), 27, 1.25, 32, 11500)
(3, datetime.date(2012, 1, 12), datetime.time(10, 45), -17, -2.26, -12, 44700)
Datarow(id=1, date=datetime.date(2012, 1, 10), time=datetime.time(12, 35), territory_id=17, A=3.14, B=22, C=33500)
Datarow(id=2, date=datetime.date(2012, 1, 11), time=datetime.time(13, 5), territory_id=27, A=1.25, B=32, C=11500)
Datarow(id=3, date=datetime.date(2012, 1, 12), time=datetime.time(10, 45), territory_id=-17, A=-2.26, B=-12, C=44700)

What AXON looks like

AXON is eXtended Object Notation. It's a simple notation of objects, documents and data. It's also a text based serialization format in first place. It tries to combine the best of JSON, XML and YAML.

Several examples could make clear what it looks like...