Artikulate Plans for Randa

Language learning is often considered as the task of memorizing new vocabulary and understanding the new grammar rules. Yet for most, the most challenging part is to actually get used to speak the new language. This is a problem that Artikulate approaches with a simple idea: to learn the correct pronunciation of a word or even a longer phrase, the learner listens to a native speaker recording, repeats and recordings it, and finally compares both recordings to improve herself/himself with the next try.

Since a while, Artikulate is shipped in the KDE Education module. Yet it is one of the few still Qt4-based applications. Actually, this is something that should change :) Most parts of the application is already ported to Qt5/KF5, yet the whole UI is still in an experimental porting stage, due to the invasive porting changes to QtQuick2. The next step is to finally complete this porting and even to go a step further. Since some time there are already quite promising mockups for a new UI around that were discussed here.

An excellent opportunity to work in this is the upcoming sprint in Randa. But since mere coding is maybe not enough to justify, why one should travel in the middle of the alps, the plans for that week are more ambitious:

  • finish the UI port and apply the new UI layout: see above
  • prepare a first touch prototype: this is an excellent opportunity, since all the mobile experts are there
  • record some very basic course sets: currently we only have many recordings for Bengali and Polish, but many contributors with different tongths around in Randa; my goal is to have some mini-courses for the languages I need most often when traveling through Europe, and particularly tailored to the basic words needed on such trips.

Your help is Needed
Sprints like the upcoming sprint in Randa are essential to keep developers busy developing, to provide them with opportunities to discuss their projects’ next steps, and give them the chance to exchange wisdom/experience/ideas. And probably not less important, to keep the community feeling alife that KDE is all about. You can help with even a small donation. How exactly, is explained here:

Randa Fundraiser-Banner-2015

The QML State Machine

The Qt 5.4 release is coming closer and it brings a whole lot of nice things: high DPI support, Qt WebChannel, and much more. One of these very cool, yet maybe slightly inconspicuous, new features is the QML State Machine. It brings a fully featured state machine to the QML world, which is a finite automaton consisting of states, transitions that define on which event to transit from one state to another, and event handlers that are called when a stated is left or entered.

Despite its simplicity, a state machines features a very powerful tool to ease the creation and maintenance of complex QtQuick UIs. In a former post I very briefly talked about the rewrite of Rocs’ visual graph editor in QtQuick. A major problem of this rewrite was that it became (even with only basic functionality) a very complex beast to handle. To give a little bit of context first, the visual graph editor currently consists of four mutual exclusive actions that can be performed at a scene: select some and/or move the selected nodes, connect two nodes by an edge, create a new node, delete scene elements by clicking at them.

Now, I played around with rewriting this functionality by using state machines, mostly to figure out how effective state machines are for this use case. For my experiment I ported the first two of these actions, whereas activating an action simply starts a corresponding state machine. For the select-move action the state machine code looks something like this (compared to the original code that lives in the editorStateMachine branch in the Rocs repository for now, functionality is simplified a little bit to improve readability):

import QtQml.StateMachine 1.0 as DSM
...
DSM.StateMachine {
    id: dsmSelectMove
    initialState: smStateIdle
    running: true
    DSM.State {
        id: smStateIdle
        DSM.SignalTransition {
            targetState: smStateMoving
            signal: scene.onMousePressed
            guard: scene.mousePressPerformedAtNode
        }
        DSM.SignalTransition {
            targetState: smStateSelecting
            signal: scene.onMousePressed
            guard: !scene.mousePressPerformedAtNode
        }
        DSM.SignalTransition {
            signal: scene.onMouseClicked
            onTriggered: {
                scene.clearSelection()
            }
        }
    }
    DSM.State {
        id: smStateSelecting
        DSM.SignalTransition {
            signal: scene.onMousePositionChanged
            onTriggered: {
                selectionRect.visible = true
                selectionRect.to = scene.currentMousePosition
            }
        }
        DSM.SignalTransition {
            targetState: smStateIdle
            signal: sceneAction.onMouseReleased
        }
        onEntered: {
            scene.clearSelection()
            selectionRect.from = scene.lastMousePressedPosition
        }
        onExited: {
            selectionRect.visible = false
        }
    }
    DSM.State {
        id: smStateMoving
        DSM.SignalTransition {
            targetState: smStateIdle
            signal: scene.onMouseReleased
        }
        onEntered: {
            scene.startMoveSelectedNodes()
        }
        onExited: {
            scene.applyMoveSelectedNodes()
        }
    }
}

The code itself probably can live without further comments, except that scene is the component containing all elements and that the scene re-transmits all mouse events which happen there. Furthermore, the selectionRect element is a rectangle that is displayed whenever the mouse is pressed at the scene and moved in some direction (hence, a selection rectangle :) ). If you know some QtQuick you probably can estimate how this functionality is implemented in a typical way: you have to create a lot of bindings, properties that store state information, and maybe actually some QtQuick states — with the result of having the logic distributed over the whole QML file.

This problem mostly vanishes when using state machines, and furthermore, by using state transitions one can drastically improve readability. Try it yourself! And thanks to the devs who implemented this cool tool!

Rocs KF5 Progress

Main message: The Rocs frameworks branch is finally merged into the master branch (but too late for Applications 14.12 release). It has tons of changes, not only porting ones but rather refactorings and technology changes.

And now in Details: The work on the Rocs frameworks port was quite a bumpy road. Back in early spring this year, I started with an initial porting afford to KF5/Qt5, but soon it turned out that there was a chance of combining this porting with a long outstanding code refactoring of the application’s core. Before I am going into details, here some numbers of this port:

  • 835 files changed
  • 36924 insertions(+), 62342 deletions(-)
  • 379 commits

Having these numbers in mind, I also made the decision to take some more time for stabilizing the code base until the spring release and not to rush into the Applications 14.12 release. However, most work is already switched to polishing the remaining glitches and fixing the bugs that were undoubtedly introduced with these big changes. And for my personal usage, I can say, that the current master branch version is already quite usable on a daily base.

Rocs Qt5/KF5 Port

The big refactorings in a nutshell:

  1. Proper project support.
    We have a long history of at first only operating on a loose set of graph and script files, then introduced projects to (optionally) organize these files. Now the core of each Rocs session is a project, which on the one hand makes the workflow much simpler and on the other hand simplifies the code base quite a lot (now there is only one place in code that cares about project files and sessions).
  2. A scene graph based visual graph editor.
    Rocs in the 1.x versions used a graphics view for providing a visual editor for graphs. But this part in particular was a “grown” code base, containing some of the oldest code in Rocs (~7 years). With having QtQuick as a kind of successor technology for our use case, I started a complete rewrite of the editor in QtQuick. The graph editor is already usable but still contains many glitches and some features are missing, compared to the old one. This will probably be the main area of work until the spring release in 2015.
  3. A leaner graph model with better scripting API.
    One part of Rocs that is of most importance to me, as someone who uses the application to actually do research and test graph algorithms, is the graph model. Rocs has an internal representation of a graph theoretical graph, consisting of nodes, edges, dynamic properties at these elements, and a set of basic graph operations. Due to history, the graph model was never completely covered by unit tests and further, over the years, gained many (and some of them hacky) extensions: elements received types, graphs gained the option to represent different data structures, several versions of different scripting APIs were introduced… So, during this port, the graph model was completely rewritten from scratch, but with all the lessons learned over the years. And as a plus, following a test driven approach, every operation is unit tested! Having the experience of working on the model for some years, it was also possible to make it much leaner and only provide API and functionality that is actually used somewhere in the application. Together with the visual editor the graph model forms the new (but internal for now) graphtheory library.

During the next month, I will try to talk in more details about each of these points, as I think for each of them there are interesting things to tell, and that not only for Rocs developers :)

Randa report: Artikulate KF5 port (almost) done

It is the Randa-Sprint week again. If you never heard about this, then imagine a lot of KDE developers, meeting somewhere in the mid of the Swiss Alps, in a deep valley with a rather slow internet connection. These people are coming from all over the world and are here for exactly one week, to work, to discuss, and to create the future of KDE. To name only a few of the current meeting’s topics, there are people working on a KDE SDK, porting to KF5, writing the KF5 book (aka putting documentation to the KF5), reaching out for new platforms, and many more exciting things are happening here. If you want to know more about all the goods that the Randa meeting brings, you should probably have an eye on the planetkde.org posts for the next days.

My own small contribution during the last few days was working on the KF5 porting efforts in KDE Edu. And finally, the Artikulate port is mainly done: It compiles. It runs. And the core functionality is working. Nevertheless, there are still a lot of glitches, which have to be ironed out. (Especially, since I did a very bumpy porting from QtQuick1/PlasmaComponents to QtQuick2/QtQuick.Controls…) Yet, they still have to wait a little bit, since currently there is work ongoing on getting a new freshed-up layout… Interested? Then, have a look at the Forum.

Artikulate on KF5

KDE Edu speaks AppData

Just in time before the KDE SC 4.14 string freeze applies, I merged my last AppData patches and as of now: all KDE Edu applications provide AppData meta information. This means, they will be better visible in software centers like Apper or GNOME-Software.

Well, this is only a very short announcement, if you want to read more about the benefits of and ideas behind AppData and AppStream, please have a look at Ximion’s blog.

Just a personal experience note after touching those > 20 files: The auto generated files are quite helpful and provide a good start. But effectively, every file had to be adjusted by hand. This does not come from the generation scripts but from the fact that we have a multitude of application descriptions, webpages, and screenshots that are not all in sync (yet). The most common things that had to be updated in the pre-generated files were:

  • The homepage address was missing or pointing to an outdated page.
  • The screenshot URLs were broken, missing, or we provided an awfully old version somewhere at our websites (will be fixed soon or is already fixed ;)
  • The description texts were often quite out of date.

So the next step after having this up-to-date AppData files is to backport the updates to the websites. In particular, I am looking forward to re-use the AppData files to have an easier way to generate the static application data on our websites. But more to that in a later blog post…

By the way: There is currently happening a lot in KDE Edu and you can help all those people who create free and open educational software for you by supporting the upcoming sprint in Randa:

Donations can be done HERE.

Artikulate port to GStreamer-1.x is ready to merge

As an pronunciation trainer, Artikulate does not only need sound output, but also sound input. For sound output the (Linux) world is kind of simple and it only costs some lines of code to integrate Phonon or a similar framework into your application. For sound input it is trickier, especially if you do not want to work with a C-style API (when coding a C++/Qt application, C-style API feels very strange). The solution to this problem is QtGStreamer, which provides Qt-style C++ bindings for GStreamer, and hence access to the currently best multimedia framework.

Until recently, the downside in using QtGStreamer was that one was stick with GStreamer-0.10. That is especially problematic as GStreamer-0.10 is unmaintained, and in one are another way, every reported sound bug in Artikulate could be tracked down to having its root in that old GStreamer version.

But people did some awesome work, and the qtgstreamer/master branch finally supports GStreamer-1.x. And since today, also Artikulate is ready and in the artikulate/gstreamer-1.0-port branch, you can find a fully working port of Artikulate, which uses current qtgstreamer/master. It is well tested and will directly be merged into master, once a new QtGStreamer version is published.

Now, that it is only a matter of time that all our sound issues are fixed, the next step is to look forward for porting to Qt5. For Artikulate and any other KDE Edu application it will be also a major step in being able to reach a wider audience by our applications, by being able to support many more platforms.

If you want to support this and many other efforts, here is your chance:

The Randa sprint is looking for sponsors!

Interaction Design for Users

If you followed my last blog posts or those by Oindrila or Magdalena, you could see how Artikulate evolved over the last nine months: We started with a simple “development UI”, meaning a quite arbitrary puzzle of buttons and lists at the screen. When features were implemented, also the UI started to reflect that by receiving more buttons, structure and complexity. In recent times we started to look at workflows and tried to model them in a best afford way. Our focus was on important workflows and interactions, where “important” was understood from our point of view as developers. — So we implicitly assumed that our users’ workflows match with our own ones.

Yet, this approach is wrong. When designing an application for a user, we should focus at his/her expectations (which are not necessarily ours).

Hence, we started to rethink our design process from scratch. Our goal is to first understand who our users are and what they want, before designing for them. For this we make use of a widely adopted technique to identify our user’s interactions with the program: We define personas and evaluating the personas’ interactions with the program. Here a persona is a character quite similar to a real human being. They are described in such detail that we are able to figure out how they interact with our program. So we can set as new focus for our interaction design that we want to fulfill the goals of our personas — and by doing so, we expect to meet the expectations of all of our target audience.

This work already started at last Akademy, where I presented a set of personas that in my opinion characterize our target user groups of language learning software in KDE Edu. Essentially these are:

  • Tina, a German Business Consultant: She travels a lot and has to use different languages at work. Her proficiency in languages is also connected to her career opportunities.
  • Gan, a just retired US American history teacher: He just figured out how much free time he has and he plans to use much of this time to make trips to different places in the world. But he does not want to be a typical tourist, he wants to be informed about the country and even to be able to express basic wishes and questions in the country’s language (like ordering food). Yet, besides English he only knows some words of French that he learned 30 years ago at school. Currently he is planning a trip to Spain.
  • Camille, a 14 years old French pupil, living in Paris: She has to learn English at school, and parents expect that from her. Improving her language skills essentially means the benefit of getting better grades. Though besides what her teacher gives her to learn, her ambitions are limited.

Of course one can think of more personas and also slightly different characteristics of these. But that is not the goal. These personas already embody the key characteristics of all of our (current) target users. So our task is now to design software that really fits the needs and goals of Tina, Gan, and Camille.

During the current Edu sprint (that happened the last six days in A Coruña, Spain) we already walked through all of our KDE Edu language learning applications (especially, KLettres, KWordQuiz, Parley, Kiten, Artikulate) and discussed how happy our personas are with them and what they don’t like. What we found out were actually some interesting insights where workflows are unnecessarily hard or confusing and where functions where missing, especially at places where we never expected them beforehand… Specifically for Artikulate, I was quite surprised that actually none of our three personas wants to select courses or wants to download them. Rather they want to only select a language and start training their pronunciations for it, either by selecting a problematic sound our a scenario. But how the program provides a (especially if there is only one available) is not of interest for them. (This does not mean that we disallow selecting and downloading courses in the future, but we will put those options to a place where it does not complicates our personas’ workflows.)

For Artikulate we already started at the sprint to make quite drastic changes to the UI with the goal to satisfy Tina, Gan, and Camille. How that will look like will be visible in some weeks :)

Follow

Get every new post delivered to your Inbox.