venerdì 3 aprile 2015

My experience in Open Source Stenotyping - By Paulo Paniago

Paulo Paniago is a long time Plover user, and what he accomplished is amazing: he wrote a Portoguese dictionary from scratch and modified Plover to accomodate the different layout, and he did all this while learning to stenotype.
We are happy to share his experience with you, check out his article below!

As I am proud to say, and I always repeat it to exhaustion: I believe I am the first human ever to learn to stenotype in Portuguese, from an open source project.
My story inevitably mixes with Plover, the program from which I’ve learned to stenotype. Founded by Mirabai Knight and Josh Lifton, Plover is the first open source steno program and now an umbrella project for other free public license steno related software and hardwaresuch as a clone project for android (Steno Keyboard by Brent Nesbitt), interactive lessons with games for speed up learning,  (QwertySteno by Mike Neale), didactic material (Learn Plover by Zack Brown) and printable 3D keyboard (Stenoboard by Emanuele Caruso).

As what lead to my interest in steno, I can say that because of an isolated lifestyle, I developed an unusual urge for writing. As the activity progressed into passion, I became interested in learning faster methods of writing, idealistically, to write as if I was just “telling something to someone”.
That’s why, more than twelve years ago, I wanted to learn to stenotype. While, fortunately, I could get my hands on a textbook, I gave up on it because of prices in classes and equipment. Since I am hard of hearing, I couldn’t possibly justify those expenses at the time – to become a captionist? If it weren’t for that limitation, even thou I never ambitioned a career in the area, I could have a convincing excuse of learning it “just in case” I needed a different job.
When I discovered Plover, a free program that enabled me to learn steno with available devices, although even interested for so long in steno, it wasn’t without resistance that I decided to start. That is because I had just invested great amount of time and energy learning to type in another unconventional keyboard – a brasilian “dvorak” styled layout called br nativo, which was developed by Brazilian programmer Ari Caldeira.

A dvorak keyboard functions the same way as conventional keyboards – known as “qwerty” – with the only difference being the order of the letters:

This is Ari Caldeira’s br nativo, the dvorak layout I learned before steno

As an over simplification, the main advantage of dvorak keyboards, when compared to qwerty, is that the most frequently used letters are in accessible positions, for instance, having all the vowels in the middle column. Whereas there main difference between qwerty and stenotyping, is that in the stenomachine you press multiple buttons at once.
It may sound difficult, but there are abbreviation principles that enable you to press fewer keys than what would correspond to in conventional typing. This way, the program uses an internal “dictionary” to convert the abbreviated “strokes” into common words. Generally speaking, you learn the logics behind your steno theory and than condition yourself to type by “sound” and/or memorization.
Well, and here is where my struggles began: Since the steno system is so dependent on the arrangement of keys – the keyboard layout – that also meant that this organization should be optimized to work better for their native language or theory.
Unfortunately for me, phonetic dissimilarities between English and Portuguese lead to different steno keyboard layouts.
 
 (Brazilian steno keyboard)


As you can see, many letters are arranged in different order.


So, if I wanted to begin my training, I would have to change the keyboard layout inside plover program. It would have been so much easier if, in another scenario, the keyboards in both languages where the same. If that was the case, and since my English is reasonable, I could have learned Plover for English, absorbed that basic keyboard layout into muscle memory, and than only later worry about adapting it to Portuguese.
After a lot of resistance, I only decided to learn steno simply because I knew that twenty years from now, I would regret if I didn’t. When I began to try it in my computer by “arpeggiating” I got so excited that I brought a gaming keyboard even before dealing with the problem of keyboard layouts.
Since Plover has the capacity to turn an inexpensive n-key rollover keyboard into a steno machine, I brought the cheapest compatible device at the time: the Microsoft’s SideWinder, which was the first gaming keyboard to have anti-ghosting technology, even though being actuated by membrane circuits. Generally speaking, all other keyboards with n-key rollover were more expensive because they had mechanical switches designed to give “tactic” feedback similar to that of ancient typewriters.
By n-key rollover or anti-ghosting technology, I mean that the keyboard has capacity to recognize many keys pressed at once – that basic feature of any stenomachine. Whereas professional machines cost a few thousand dollars, that keyboard was conveniently affordable.
Although gaming keyboards with their native “qwerty” alignment are not accurate, fast or ergonomic as traditional stenomachines to be classified as professional equipment, they can be efficiently used as a “student” tool to enter the steno world.
Even thou later hardware solutions derived from the niche Plover created could be placed somewhere in between gaming keyboards and professional stenotyping machines – such as the Stenoboard, with parallel column keys and split keyboard design – I could consider myself equipped to start my training
So, thanks to Plover, I began to learn steno two year ago. I only had the sidewinder, a textbook, Plover configured for English, and some guidance from Mirabai, the stenographer behind Plover project, in her Aviary forum.  The Aviary forum was the place, in that project, for beginners to post their thoughts, frustrations, expectations, and mainly to ask for help. But there was no one else with any request vaguely similar to mine: I knew close to nothing about stenotyping and wanted to learn a steno theory from a project that was not even adapted for my language.

About my textbook, I would like to add that, although vital for me to learn steno, it was only a textbook of exercises – we call them “apostila”. It had brief explanations in each lesson followed by many exercises to apprehend them. Unfortunately for me, the exercises came with no answers. Those would be in the machine itself configured as the student’s dictionary.
I remember those were though times. One day I had an idea that couldn’t believe I haven’t thought of it before: I could use the program with the “wrong” keyboard layout, but with the right typing and right transcriptions – all coherent with my Portuguese steno theory. This way, the steno codes would all be wrong, because the machine was configured to use the US keyboard layout, but it would still be useful as a stenomachine. Although I knew the idea worked, it would be awful, since I would not be able to read my own dictionary.
The  “dictionary”, in steno dialect, is basically a simple but extremely long “binomial” list of code versus meaning. It is the file that the program uses to transcribe the steno notes into common words.
If I always typed correctly and never changed any basic line in my theory, that idea could work indefinitely. But an indecipherable dictionary would be a limiting factor if I eventually decided to change rules or make big corrections.
Anyway, I was so obsessed with at least starting my training that I decided to put my plan in practice, even neglecting some of Mirabai’s advices in contrary:  “Not sure that's the way you want to go, because it'll make it very difficult to understand the Portuguese steno outlines on an intuitive level; everything will be "sounded out" in a non-phonetic, illogical way, because it'll be using English layout rather than Portuguese”
In those days, Plover didn’t had an “add translation” window so I had to turn the program off, open the dictionary file and manually type the words missing. Looking back, even thou I didn’t recognized at the time, I can’t believe I had the courage to start my dictionary from a blank page.
To illustrate how the dictionary file worked and how I could write it with “the wrong steno codes but the right typing and right transcription”, let me give you an example:

Let’s say I wanted to define the word “bar” in my dictionary file:
In my Brazilian Portuguese theory “bar” would be written with the stroke: “TPAR”
So it would appear “binomially” this way in the text editor of that file:
"TPAR": "bar",
 Although with the American layout, the same word pressing the same physical keys, would have to be written this way:
"KWAF": "bar",
So, to accomplish my plan, I would have to define the whole dictionary file in a language that I could not understand.
I remember some tricks I used to make the process of writing that strange dictionary easier, but to which I don’t have the proper ability to explain. Basically, after some installations and configurations I set two layouts using a program called Ukelele – Ukelele is a program designed to make “qwertys” or “dvoraks” keyboard layouts in Mac OS X system, and has nothing to do with steno –  it was actually the way I found to configure Ari Caldeira’s “br nativo” in my notebook, long ago.
So than, after the two keyboard layouts was configured, I closed my eyes and pressed each key, one at a time, in the order they should appear in the steno paper. While with the eyes closed, I would be “thinking” that I was typing “TPAR”, but because of the configuration I did, it would appear as “KWAF” in the file. Than I opened my eyes, changed back the layout, and wrote the corresponding word: “bar”.
It was laborious to say the least. Naturally I made many mistakes and countless corrections in the process.
At that time, after months of tedious and insecure blind typing to the dictionary, I was hoping for some steno enthusiast to contribute with a feature to change layouts in Plover. And also, since I couldn’t find a programmer to do the job for me, I finally decided to change the layout myself, even thou I knew nothing about programming and was even afraid to look at files.
While guided by some of Mirabai’s tips, I searched the Internet for people who made similar changes in Plover. That ended up giving me two names: Aziz Yemloul and Krzysztof Smirnow.
 Both made profound modifications to Plover’s keyboard layout because of linguistic issues, and fortunately for me, both posted their procedures in internet forums.
Aziz Yemloul changed Plover in comprehensive aspects to adapt it to the French steno system. He ought to do much more than what I intended because, since the French machine was invented independently from their North American counterpart, it uses a physically different keyboard, called “Grandjean” or “Continental”:

Although both systems converged to many similarities in practice, as you can see, the continental steno keyboard is angled and symmetrical. By symmetrical, I mean that it uses the same number of keys for both hands.

 Whereas Yemloul made a complete project – which included not only a physically different keyboard, but also full orthographic rules to manage the French dictionary – what Krzysztof Smirnow wanted was even lengthier.
Smirnow was willing to adapt Plover for Polish language. The complication to his goal was that there has never been any steno theory for that language. So, he would not only have to create a the theory himself, but also decide through all it’s technical specifications, from keyboard layout to physical arrangement to frequencies relating terminations and prefixes.
 To solve orthographic issues, he envisioned a physically different keyboard. The reason, according to him, was that because of complexities in Polish language, he would need a keyboard with more keys. This way, he was looking forward for another symmetrical organization. But his keyboard, instead of looking like the French “continental”, would be similar to the American steno machines, although with four keys, instead of one, for the little left finger.
 Unfortunately for him, and following his explanation, Plover was made in such a way that increasing the number of keys and defining particular characters from Polish language would not be feasible without major modifications to the program’s base – which would then require rebuilding Plover almost from ground. Consequently, as of the last time I read, Smirnow was still dealing with those obstacles.
So, reading their posts about how they managed to change the keyboard layouts, that all looked like fragmented answers for me, since I understand basically nothing about programming. Even combining their topics with suggestions from a Brazilian professional pen shorthand writer who emailed me telling how he made those changes it in his linux system, I still didn’t knew what to do.
About this Brazilian, back in those days, he asked me not to mention his name. That was because, here in Brazil, court reporting is taken by pen shorthand writers called “taquígrafos”. Since there is resistance and natural hostility from established professionals intimidated by the “rise of the machines”, he feared for prejudice from his colleges, if they ever discovered he was after steno. Unfortunately – or fortunately for me, since it reinforces my claim as the “first human ever to learn to stenotype in Portuguese from an open source project” – I believe he quit steno in that very beginning, since he vanished from any of our correspondence.
I had even visited a Brazilian university searching for students acknowledged in python programming to help me change the layouts, but to make an already long story short; after a weekend only searching files in Plover, I finally found what I guessed was the configuration to the keyboard layout.
They were written in a strange language and I couldn’t believe a programmer was able to deal with them. Without having any idea of what I was doing, I changed one letter and tested. Then corrected it back, changed another letter and tested. Repeated that process again and again until it finally gave a result by replacing an output letter from the keyboard layout.
I couldn’t believe that was actually happening.
Only latter, I discovered that those were the “compiled files”. Compiled files, with the extension “.pyc”, are generated by the program itself, in a language used the computer to process and are not meant to be readable by human.
 Again, I couldn’t believe it, when I had correctly changed the whole layout myself – It looked like magic when I saw my keyboard working as a Brazilian steno machine. I guess Mirabai even got embarrassed when I posted in the Aviary: “I finally changed the Plover keyboard myself… I had just to change the letters in two files” just a few messages after she wrote: “I wish I had the expertise to do it for you”.
Since then, with a sidewinder as my steno machine, I had consistently trained one hour a day and it took me nine months just to cover the lessons in my textbook. I don’t know if I was slow because I had to write the dictionary from blank, because of insecurity, since I never had a clear idea if what I was learning was in the same line as my steno theory, or if I am simply a slow learner.
I don’t bother much being the last option, of a slow learner; at least that puts me in the category of “persistent”.
I’ve started my steno training in May 2013 and after one year, even though slow, I was already using steno for everything.
My strategy for grounding up my dictionary was to be conservative. Even to create my own briefs I generally only defined a word in my dictionary when I was confident of the theory behind it. In the other moments I relied on spelling. I had set my spelling, in plover, to resemble as much as possible my “qwerty” typing (br nativo)  – even at the cost of sacrificing some common words and briefs. I use it for about ten percent of my typing in Portuguese. – I spell, as I am doing here, most of the times when typing in English.
Now, after almost two years since that fatigued beginning, I believe I have about one third of the speeches speed, which is already a bit faster and a lot more comfortable then my conventional typing was – with the additional motivation that, while doing so, I’ll get faster and faster, and, what is more important for me, I’ll get a fluency closer to the speed of my thoughts.

Paulo Paniago

9 commenti:

  1. Olá, Paulo! Tudo bem?

    Gostaria de utilizar o Plover no meu dia a dia como taquígrafo. Você pode me orientar quanto ao que posso precisar para começar? Não tenho grana para comprar um teclado steno, mas, parece que utilizando o Plover ele "emula" um steno keyboard.
    Eu fiz umas frases aqui, mas preciso de mais informações de um brasileiro, como você, que já passou pelas aventuras de tentar utilizar o teclado steno num teclado normal (qwert).

    Forte abraço!

    RispondiElimina
  2. oi Eduardo, desculpe a demora, eu não tinha olhado essa página. Mas, sim eu acho que posso te ajudar. Você pode me procurar no bate-papo do Plover? Lá a gente troca os e-mails também. você pode começar um tópico como "Plover para o português". É até bom que vai mostrar para os programadores do Plover que tem mais gente interessado em estenotipia para o português.
    https://groups.google.com/forum/#!forum/ploversteno
    abraços,
    Paulo Paniago!

    RispondiElimina
    Risposte
    1. Valeu, Paulo. Vou querer começar com tudo agora. Preciso de ajuda para começar.
      Vamos nos falando.

      Elimina
  3. Questo commento è stato eliminato dall'autore.

    RispondiElimina
  4. Olá Paulo Paniago, tudo bem! Já sou um estenotipista formado pelo TJSP. Lá utilizo um sistema proprietário, e por isso estou impedido de trabalhar fora daquele órgão. Estou interessado em usar o Plover, não fosse a limitação em relação ao Layout para português. Se tiver novidades estou por aí.

    RispondiElimina

  5. Olá Fernando, desculpe não ter visto a sua mensagem aqui. Eu não costumo entrar nessa página. Mas gostaria de repetir para você a mesma resposta que dei ao Eduardo Ramos. Você poderia me procurar no bate papo do plover Google groups? Eu agradeceria se você abrisse um novo tópico pedindo ajuda para a conversão do plover para português, para novamente, mostrar que tem mais brasileiros interessados na estenotipia. Eu tenho a impressão de que a nova versão do Plover aceitar troca de layout, mas não testei ainda. Eu ainda estou usando o plover 2.5.8, aquele que eu tinha conseguido alterar o layout há tanto tempo atrás. https://groups.google.com/forum/#!forum/ploversteno

    RispondiElimina
  6. aqui segue o link do bate papo do plover google groups
    https://groups.google.com/forum/#!forum/ploversteno

    RispondiElimina
  7. pode escrever lá em português mesmo...

    RispondiElimina
  8. Excellent read. And inspiring too.

    I'm in the beginning stages to develop stenotype in my language. And as such would appreciate your advice. Hope to hear from you. And wish you success in your endeavours. I can be reached at: nor.leban@gmail.com

    RispondiElimina