How to Convert a Project to REUSE Compatible License Statements?

This blog post provides a step-by-step example about how the conversion of a project to REUSE compatible license statements is done in practice. For my setup, I have a readily configured kdesrc-build environment.

First, I get out the most recent source code if the project I want to convert. For this tutorial, I use KTurtle, which is a nice and small application from KDE Education with just about 200 files.

Then I obtain the latest version of licensedigger and compile it:

kdesrc-build licensedigger

First I do a dry run to get an impression about how well the licenses are detected in KTurtle. But, well, it looks really bad:

$ /opt/kde/build/playground/sdk/licensedigger/licensedigger --dry kturtle/
Digging recursively all files in directory: "kturtle/"
= LICENSE DETECTION OVERVIEW =
"kturtle/CMakeLists.txt" --> "UNKNOWN-LICENSE"
"kturtle/doc/CMakeLists.txt" --> "UNKNOWN-LICENSE"
"kturtle/icons/CMakeLists.txt" --> "UNKNOWN-LICENSE"
"kturtle/org.kde.kturtle.appdata.xml" --> "UNKNOWN-LICENSE"
"kturtle/spec/assert_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/boolean_operator_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/empty_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/expression_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/for_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/if_else_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/kill_kturtle.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/learn_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/math_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/number_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/repeat_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/scope_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/spec_helper.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/start_kturtle.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/string_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/spec/variable_assignment_spec.rb" --> "UNKNOWN-LICENSE"
"kturtle/src/CMakeLists.txt" --> "UNKNOWN-LICENSE"
"kturtle/src/Messages.sh" --> "UNKNOWN-LICENSE"
"kturtle/src/canvas.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/canvas.h" --> "UNKNOWN-LICENSE"
"kturtle/src/colorpicker.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/colorpicker.h" --> "UNKNOWN-LICENSE"
"kturtle/src/console.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/console.h" --> "UNKNOWN-LICENSE"
"kturtle/src/directiondialog.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/directiondialog.h" --> "UNKNOWN-LICENSE"
"kturtle/src/editor.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/editor.h" --> "UNKNOWN-LICENSE"
"kturtle/src/errordialog.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/errordialog.h" --> "UNKNOWN-LICENSE"
"kturtle/src/highlighter.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/highlighter.h" --> "UNKNOWN-LICENSE"
"kturtle/src/inspector.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/inspector.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/dbus_adaptor_generator.sh" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/definitions.rb" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/echoer.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/echoer.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/errormsg.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/errormsg.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/executer.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/executer.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/generate.rb" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/interpreter.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/interpreter.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/org.kde.kturtle.Interpreter.xml" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/parser.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/parser.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/token.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/token.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/tokenizer.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/tokenizer.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/translator.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/translator.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/treenode.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/treenode.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/value.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreter/value.h" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreteradaptor.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/interpreteradaptor.h" --> "UNKNOWN-LICENSE"
"kturtle/src/main.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/mainwindow.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/mainwindow.h" --> "UNKNOWN-LICENSE"
"kturtle/src/sprite.cpp" --> "UNKNOWN-LICENSE"
"kturtle/src/sprite.h" --> "UNKNOWN-LICENSE"
Undetected files: 69 (total: 69)

What we get from this output is that apparently no license header is detected. This suspiciously looks like that we find a new kind of license header texts, for which licensedigger was not trained yet. Thus, I arbitrarily open one of the failing files, let’s say “kturtle/src/sprite.h”, and have a look at the header. The stated license header itself looks quite sane and clearly translates to the SPDX identifier “GPL-2.0-or-later:

Copyright (C) 2003-2008 Cies Breijs

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public
License as published by the Free Software Foundation; either
version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
Boston, MA 02110-1301, USA.

Thus, I first try adding the missing license header text to the licensedigger database (see the licensedigger README.md file for more explanation) or the commit with which I did this, which is this commit. But well, and this is surprising, when testing the license header detection by running the unit tests with “make test”, the header still is not detected… This is surprising. It turns out that tabulators for indenting license header texts are not supported yet. This is easy to fix, which is done in a follow-up commit. But if you ever land in a similar problem, just create an issue and point me to the source file that makes headaches! (Such problems are really a rare exception though!)

Now I run licensedigger again and check if there are files remaining which not correctly detected license headers, which is the case for me and I repeat the above steps. Finally, I get to the point where all stated licenses (and we cannot convert unstated ones; recovering missing license statements is a complete different topic) are correctly detected.

Finally, I run license digger in its conversion mode. Note that you can also run licensedigger many times on an already converted codebase and nothing bad happens; I simply prefer to distinguish between dry and conversion runs.

When that is done, the last remaining steps are:

  1. Review the changes licensedigger did to your source files (I like to use the command line tool “tig” for this, but there are many options).
    1. I commit the changes to the source files to have a base-line for possible manual edits.
    2. For KTurtle I actually manually formatted the license headers after the conversion to remove the tabulators. Unfortunately, there is not tooling yet for auto-format license headers (patches welcome 😉 )
  2. I add the new LICENSES/ directory to the Git repository, which contains the canonical license texts according to the REUSE specification. Moreover, I remove the now obsolete COPYING file, because that license text now is in the LICENSES directory.
  3. Finally, I can create a merge-request on invent.kde.org, because it is always good to let somebody else check your work.

And as soon as it is approved and merged, another repository enters the shiny new world of REUSE compatible license statements. For more background about REUSE compatible license statements and the route we are following in KDE, you might want to have a look at the licensing howto wiki page.