Soletta IoT Framework

Every once in a while there comes the will to blog again, and this is that time. Lots of things happened since the last post here, jeez, the first of them being I joined Intel ever since. It’s been fun to work there, by the way. The thing is all of my projects were work on proprietary (and non-disclosable) software for a time, thus the silence.

I had the opportunity of working on a whole visual identity for a Tizen-based smartphone stack, which was awesome. I also had the opportunity of working on an Android fork that implemented multi-windowing. It was not as awesome, because Java gives me the cramps, but I got to learn that system’s internals and learning is always fun.

The next project I jumped onto was really exciting, though (now open source!). I was involved since the very beginning in Soletta‘s design and coding. One thing we had in mind, when starting that in this already fragmented IoT ecosystem we got, is to really make a difference in completeness and ease of use for developers of all backgrounds. There are a lot of claimed IoT solutions out there, really. But what we see is that the majority of them are too focused on a small subset of what an IoT node should be able to accomplish (communications or I/O), but not on everything altogether. Also, the things we saw out there were not portable at all. We wanted porting Soletta to different platforms a breeze from the very beginning.

So, we started our journey. In order to accomplish the ease-of-use part of our proposal, Soletta based its API in a main loop, so that it’s easy for users to hook on events by simply giving callback functions and never having to deal with threads and the like. We started with glib’s main loop as a back-end, but now we got one of our own to share among POSIX systems and we made sure writing implementations for the main loop API is straight-forward for any system (glib is still selectable as a main loop implementation, by the way).

We started with one of the main parts an IoT API must have, then—I/O. We implemented the I/O nodes a program could have by having diverse I/O classes they would inherit from, and the user was supposed to connect nodes with compatible I/O classes together, so they would exchange data.

Early on that work we realized that what we were doing was very much alike J. Paul Morrison’s Flow-based Programming, and that we could benefit from that domain-specific language to have an even more straight-forward and easy way for developers to use Soletta. So we exchanged the APIs and models we got to totally support the FBP model (we now got nodes as in FBP “black boxes” and ports and connections of them). We built a FBP parser and, with that, there was now two ways in which a user could build and run Soletta programs:

  • By using the C API directly and compiling or
  • By writing their logic in FBP and using the new sol-fbp-runner tool to run it

We did not stop there, though. Thinking back of the portability promise, we made a sol-fbp-generator tool, which would take FBP input and output equivalent C code for that. This came to aid on real IoT platforms where runnable programs do not exist and everything must be compiled and linked into a final image. Average developers still don’t have to know very deeply any details of IoT operating systems that way with Soletta—they simply write their flow, with the nodes they need, and they are good to go. One important thing is that Soletta’s flow layer is not mandatory. If the constraints dictate so, one can always write Soletta applications using the rest of its C API directly and skipping the overhead (which is already really low) of the flow.

Here is what a FBP sample looks like. One declares nodes by giving them a name (timer is the name we chose for a timer node), followed by parenthesis, the node type’s name, and (optionally) colon and the node’s instance parameters. In that case, we got a timer with a 1000 milliseconds timeout. The next important things are port nodes. We must connect output ports (always written after a node’s reference) to input ports (written before node references). There, we connect the timer’s OUT port to a toggle’s IN port. The data that flows between nodes is called packets in Soletta. Ports’ supported packet types must match, so that their connection is possible. In that case, the timer outputs the empty packet type (which is just a “pulse”), while the IN port of the toggle node expects any packet type (again, it just needs a “pulse”), so they are good as a match for a connection. The toggle node belongs in the boolean family of nodes: it will keep an internal boolean state and ouput (and change) that state every time a pulse comes into its input port. Thus, the example will just print two boolean values that will change every 1 second.

By using FBP to develop with Soletta, one is able to even make use of node meta types, where there’s even more freedom of behavior customization. As of now, Soletta exposes the JavaScript meta type, where you declare a node’s input/output ports as well as the logic gluing the former to the latter all in JavaScript, directly in the FBP file. Also, there are the meta HTTP (both client and server) nodes, where one can serve and consume a custom packet they declare directly in the flow as well.

That boolean valuen to the console sample was just a dumb example of Soletta’s capabilities, naturally. Take a look at all the nodes it has, as of now. As you can see, we’re very far from that I/O-only beginning. We’ve got nodes doing all types of communication: bluetooth, CoAP, HTTP, LWM2M, MAVLink, MQTT, OIC, etc. On I/O we cover basically everything that’s relevant for IoT: analog to digital converters, GPIO, I²C, PWM, SPI, UART. We also handle persistence/storage on a various models (EFI variables, file systems and flash-memories), have Linux IIO sensors exposed and much more. Take a look at our API for yourself.

Our ease of porting effort has proved itself very good as well. Soletta will now run on any of the following systems:

It’s for real, any FBP program (with nodes having implementations on all of them) will work on each of those systems out of the box (also architecture independent). With around 100kb of final image (operating system included), we have functional OIC/OCF nodes running on very resource constrained SoCs, like the Arduino 101 or the Atmel SAM R21 Xplained Pro.

It’s been a fun journey (Soletta is one and a half years old now), and we’re finally calling it a version 1 soon! I hope all you developers in the search of solution for your IoT products have the chance to evaluate Soletta and give us feedback. There are a lot of things about Soletta and its infrastructure I did not talk about (it will be released as a behemoth already, it’s huge in what it encompasses!), so please take a look on all Soletta’s ecosystem as well.

Elementary Prefs

So, the work on Elementary, Enlightenment project’s toolkit, kept going this year. Last I blogged here, I was on my task of rewriting the entire widget set as to factorize common widget patterns in classes and create a widget tree.

So, EFL 1.7 release came and just 2 or 3 widgets didn’t make it to the conversion, up to that time. No big deal, because:

  • This next release is going to be much more awesome, because of the tons of new features (and bug corrections) coming in.
  • People just changed the inheritance infrastructure out there in Elementary.

Yep. They didn’t change the architecture/pattern decisions I made, they just exchanged the classes infrastructure from Evas Smart Objects to Eo — a new library (yep, they keep growing forever) implementing an object system. I’ll leave to that library authors a better post about the matter. This was just a background on what happened since my last post, but let’s talk about a new feature, in Elementary, I was involved with after that — Elementary Prefs.

Prefs is a new widget, along with its infrastructure, providing a mechanism to store and access configuration data, with that access being (if the user wants) in a graphical way. Yep, this is like Android Preferences, but in a EFL way:)

We designed an Edje-like (EDC-like) declarative language to describe the configuration data to be stored (and retrieved). Its composed by data items, each bound to some type. Items are grouped in pages, that can group other pages themselves. We called these files EPC (Elementary Prefs Data Collections), since they can store one or more pages of items. Here is a valid EPC file:


collection
{
   page
   {
      name: "main";
      version: 1;
      title: "Preferences Widget";
      subtitle: "Test Application";
      widget: "elm/vertical_box";

      items {
         item {
            name: "age";
            type: INT;
            label: "Your age:";
            editable: 1;

            int {
               default: 18;
               min: 0;
               max: 150;
            }
         }

         item {
            name: "save";
            type: SAVE;
            label: "Save";
         }

         item {
            name: "reset";
            type: RESET;
            label: "Reset";
         }

         item {
            name: "subp1";
            type: PAGE;
            label: "Sub Page 1";
            source: "subp";
         }

         item {
            name: "subp2";
            type: PAGE;
            label: "Sub Page 2";
            source: "subp";
         }
      }
   }

   page {
      name: "subp";
      widget: "elm/vertical_frame";

      items {
         item {
            name: "subint1";
            label: "Sub Int 1";
            type: INT;

            int {
               min: 0;
               max: 100;
            }
         }

         item {
            name: "subint2";
            label: "Sub Int 2";
            type: INT;

            int {
               min: 0;
               max: 100;
            }
         }

         item {
            name: "subsubpage";
            type: PAGE;
            label: "A Sub Sub Page";
            source: "inner";
         }
      }
   }

   page {
      name: "inner";
      widget: "elm/horizontal_box";

      items {
         item {
            name: "subfloat1";
            label: "A Sub Float 1";
            type: FLOAT;

            float {
               min: -10;
               max: 10;
            }
         }

         item {
            name: "subfloat2";
            label: "A Sub Float 2";
            type: FLOAT;

            float {
               min: -10;
               max: 10;
            }
         }
      }
   }
}

Here we declare 3 pages of data items. Note that we nest them inside each other with the SOURCE-typed item. It happens that “main” will be chosen as the parent page, and because it sources “subp” twice, we’ll see its items replicated twice on the UI counterpart of this prefs declaration. Now what would that be?

Prefs items are bound to Elementary widgets, when “realized”. We have two entities dealing with prefs data — the prefs widget, which displays that data in a visual form, and the prefs_data handle, used to save and restore back values of prefs items set by an user.

Here’s the common usage:


prefs_data =
  elm_prefs_data_new("./preferences.cfg", "main",
                     EET_FILE_MODE_READ_WRITE);

elm_prefs_file_set(prefs, "./preferences.epb",
                   "main");
elm_prefs_data_set(prefs, prefs_data);

The .epb file is a binary form of an EPC one (just like the one seen above), which we get by using the elm_prefs_cc compiler on it. This is meant to act as “system” preferences — shared by all users.

The .cfg one, on its turn, is also a binary file with prefs data. It will store values for each item of the .epb one, but just the values (no meta-data, like minimum value allowed and such). This is meant to be user preferences on this data set, i.e., values specific to the user.

In short, the former gives life to a prefs widget, which is visual, and the latter loads in it the user values back (saving them back too). For instance, each realization of the “subp” page in this collection will have its values saved separately in the .cfg file.

A screenshot of that file, when live on a prefs widget, would look just like this:

Elementary Prefs Widget
Elementary Prefs Widget

Note that most of the values have been played with in the UI, making them different from the defaults, declared in the .epb file.

As you can see, items are bound to Elementary widgets, and one can even choose widgets to represent items other than the defaults (look at the choices for pages, for example). Elementary now ships with a set of available widgets for each item type, but one can also extend these sets via Elementary modules, with the prefs signature. This is a more advanced usage, in the future I can talk more about this.

So, this is it. Expect it to hit your homes soon, in Elementary’s next release, and enjoy it (right now, if you dare to build from unstable trees).

This was a team work by Murilo Belluzzo, Ricardo de Almeida and yours truly. Gustavo Barbieri was also of key importance, coming up with most of the design.

New widgets hierarchy on Elementary

It’s been a while since I last posted here, but my work with and for the EFL (Enlightenment Foundation Libraries) has not stopped at all during this time!

The EFL have had a lot more momentum these days, since Tizen 1.0 was launched, having these libraries as important building blocks of the whole architecture of this system.

My current work has been on an important part of that stack — the native widgets library. Note that despite Tizen 1.0 being all about web apps, future releases will open up for native applications, as it was announced at the last Tizen Developer Conference, which took place at San Francisco, California.

I had the chance to attend to this Tizen conference myself, thanks to my employer ProFUSION Embedded Systems. In our stand, people were able to know our services and see, live, lots of demonstrations of native Tizen applications with rich UIs. The beauty of those UIs were made possible in great part by the use of Elementary.

Elementary is a nice widget set library, which grew fast to the current state where we got lots of widgets and nice UI-infrastructure tools. You can have an idea of the library by looking at its documentation web page.

One of the drawbacks of Elementary, because it grew that fast, is that it had no notion of widget inheritance and code reuse on the widgets base — all of them where flat, inheriting from a same internal “widget” class.

My current work has been to rework all of Elementary internals, so that we create an intelligent widget inheritance schema. I had to analyse the purpose and behavior of each widget in order to categorize them and come up with class patterns.

With that, we aim to gain a lot of code reuse and ease of maintainability. Besides, we can use commonly namespaced API calls to act on widgets belonging to a same class. What we had previously was a messy “hooks” system, in which the developer of a widget had to hook-up the widget to whichever feature it would support, and all the features were exposed and accessed in the API via general elm_object_<feature>() calls. Those would work on widgets that supported them and fail silently on the others. Now we’ll be much cleaner on what each widget supports, in syntax too.

In a not distant future, we’ll have a definitive way (present on Elementary’s API) to extend and create widgets by Elementary users, at run time. My work already provides that, but the API on creating new widgets is still subject to changes — consider it unstable.

The current state of my work on that hierarchy can be seen on the following picture:

I’ve been working hard to have all of the missing widgets (which are few) on this picture too. If you have any suggestions or doubts on this work, just ping me at freenode (“glima”) or email me. I hope you all enjoy the fruits of it:)

Finally, I must say this work on Elementary is being sponsored by Samsung Electronics. Thank you for your great effort on Linux and Tizen:)

Emacs-uncrustify updates

Emacs-uncrustify emacs extension had some small but cool contributions by Gordon Read (thanks, man!).

It has now a customization group with variables holding:

  • path to uncrustify’s configuration file
  • arguments to be passed to uncrustify
  • boolean flaging if uncrustify should be applyed on a buffer on save

We also have two different functions, now. The uncrustify one does the same — uncrustifies a region. There’s also uncrustify-buffer, now, as a shortcut for applying it to the whole buffer.

Finally, a c-mode-common-hook was added, which will apply uncrustify on the buffer if and only if that third variable is set.

Enjoy.

Enlightenment Project’s Coding Style or “If you can’t beat ’em, join ’em”

This is a quick post for all of you Enlightenment developers and users (the latter also willing to adhere to that weird coding style). The named coding style, which seems to historically have come from Rasterman’s text editor, Jed, has been recently been called for a standardization/modification by means of an uncrustify configuration file.

While the file itself is still not perfect (there are “indentation bugs” yet), I wanted to have my editor — Emacs — to help me to adapt to that thing. So, I present you with a crude first attempt at it: http://github.com/glima/Emacs-uncrustify. Patches are obviously welcome.

By the way, exciting news are emerging from E-land, so stay tuned for hot hot new stuff, soon.

Editje: Edje GUI creation for the masses

As some people might already know, Profusion has been working on a new Edje editor, intended to replace the one there was on Enlightenment’s subversion repository. I’m now participating more actively in its development and, besides that, I did help with an early version of its user manual.

It will be fixed/revamped incrementally, so that even at development stage we can have users able to do their GUI creation jobs done with it.

Stay tuned for more news on EFL applications development infrastructure soon.

The Enlightenment Foundation Libraries: A Big Picture

As part of my job activities back in August 2009, I wrote an introductory document on the Enlightenment Foundation Libraries as a client request. It is now being publicly released, hosted at Enlightenment Project’s subversion repository.

You can find a rendered version here, too, but I can’t promise how often it is going to be updated. The idea is have this document accessible by Enlightenment project’s website both in PDF and HTML formats. Ideally, both forms would be updated by a cron job or something. We’ll work on that soon.

Feel free to contribute to it, if you wish (license is Creative Commons Attribution-Share Alike 3.0) and, of course, to read it. This is one more step into making the EFL more accessible and recognized worldwide by GUI application developers.