My In-Depth Guide to Using Voice Recognition on Your Computer

In an earlier post of mine, I often mentioned that I use my voice to control my computer because of issues with my arms, and someone commented asking specifically which program I use and how I set it up. I’m going to do just that, as well as showing my thought process behind each of the steps so that you can do it yourself, whether you choose the same software or not, or even if you have entirely different symptoms than I do, you should still be able to come up with adaptations that work for you. You can treat my experience as a case study that you can apply in your own situation.

I have no affiliation, financial relationship, or any other potential conflict of interest with Nuance Communications, the creators of Dragon NaturallySpeaking, the program I use in reference the most in this article, nor do I have any conflicts of interest regarding any of the other programs and products I tried or use.

The picture for this post is my current setup since sitting hurts and standing hurts, but I was controlling my computer by voice before I had to be horizontal most of the time. I won’t be getting into how I set up this laying desk or picked out the items used to make it, I’m just going to be focusing on voice control of my computer.

When I was going through the steps below in my own life, I hadn’t divided them into official steps or categories yet. It was a much more organic and free-flowing process. But, as I reflect back on it, I see that these steps generally capture what I did, and it’s easier to remember a series of steps than it is if I was to just give you a giant stream of consciousness list. So here we go!

Step one: take stock of the situation

It started as a shooting pain that traveled from my elbow to the back of my hand and grew worse over the next several days at work and at home, particularly with computer use. I’d felt this kind of thing before, resting usually took care of it. So, that weekend, I stayed off my computer the whole time and things got better, but as soon as I restarted work and personal computer use on Monday, it came back. Over the next month, the pain never really went away.

So, the pain had just morphed from “not sure if this is something to worry about” to “yeah, probably should do something about this.” It’s an important distinction because I knew that if I got sidetracked on coming up with adaptations for every little transient pain that came up, I’d have no time to do anything else. Instead, I should focus on the ones that matter, and this one had just become one that mattered.

Once something has crossed the threshold, you likely will want to move on to the next few steps below. If something hasn’t yet crossed over that threshold, you can continue to monitor and take stock of the situation in case things change in the future, but don’t have to take action right now.

Step two: find the major culprits

At this point, the pain is actively flared for me and normal everyday activities hurt, activities that had been fine before. If I had to make a list of all activities that cause the pain to get worse, it would be a really long list and I would get overwhelmed. You can’t focus on everything all at once.

Instead, I looked at the biggest causes of it, both when it initially started hurting, and also ongoing causes that continue to flare it. For me, this is pretty easy: typing. The second biggest was using my mouse, the back of my right index finger hurt when I clicked. Since I am on my computer pretty much the entire workday, and since many of my hobbies at home include computer use, these two activities comprised the majority of the causes, so that’s what I would need to focus on.

Step three: try some simple adaptations, learn from them

Adaptations don’t have to be a major thing, and there are many adaptations you can try that are free. I had already tried one: resting from computer use over the weekend. It helped temporarily, but wasn’t sufficient to call my forearms down. I knew that resting from computer use during the week was out of the question unless I made some major adaptations, due to the nature of my job.

First, instead of clicking the mouse button with my index finger, I tried doing it with my middle finger. The pain just moved to the same spot but on my middle finger. So that one didn’t work. I also tried switching to left-hand for mouse, same deal. So that was out.

I tried using Google’s free voice software (both on my phone and within the Chrome browser on my computer), which was decently accurate but not accurate enough for my purposes. And in order to correct a word, I had to click the word and that defeated the purpose of voice typing. I was trying to get away from using my hands. So that was out.

I tried doing all of my emails on my phone using the keyboard, which didn’t flare the part of my forearms that already hurt, but a different part of my fingers started hurting due to the new repetitive use. So that was out also.

Each of these has helped a little bit, but wasn’t a good enough long-term solution. The good news was that each of the things I have tried was free and I wasn’t out any money. The other piece of good news was that even though none of these things directly solved the problem, I now had better clues for what I might be looking for in a paid solution. If I had just jumped to a paid solution, I might not have a clear understanding of what I needed.

As for my mouse, I tried using my laptop trackpad instead of the mouse, which was way worse. Good to know. Definitely want to avoid that one for me. I looked into getting a vertical mouse because the wrist position would be different, but it didn’t solve the problem of how much I would need to click, so I probably would want a way to do that by voice as well.

Step four: list out what you need and what would be nice to have

A good tool is one that does what you needed to do. If it has a bunch of features you don’t need but lacks the ones that you do, then it’s not a good tool for you. It might be great for someone else, but not for your current needs.

So, list out what features are absolutely essential: what do you need it to do? What things are so important that if they aren’t there, you wouldn’t buy a particular tool?

And, list out what features would be nice to have but not an absolute dealbreaker.

This is where the supposed “failures” in Step Three came in handy: what was lacking in each instance? From that, what did it mean I needed?

The first and most obvious thing I needed was accuracy. If my voice-to-text program wasn’t very accurate, it would be tough for me to use it to replace typing (and after all, typing is my biggest culprit and so that’s the most major thing for me to address).

I also needed to be able to edit and correct entirely by voice. That was the weakness of Google’s free option, that I had to click a word in order to correct it.

Within those two, it would be nice if the program could learn to recognize my voice and speech patterns and phrasing, but not a dealbreaker. If it was there, it would improve over time, but like I said, not a dealbreaker.

I also needed some way to control my mouse by voice. I didn’t care how, but it needed some way to do it.

Any other features it might have? Not a huge concern to me at the moment. Those could be an added bonus, but not something I was actively looking for.

Step five: look into what’s available

You could shortcut this step by asking people who have your condition (or similar) for recommendations. If it works for them, it might work for you, though of course, you would need to try it out. You could ask in person or online. You could ask medical professionals who understand you and your conditions well.

I didn’t have people like that in my life at the time, so ended up turning to the Internet. Thankfully, when looking into voice software, there were lots of reviews and ranked lists that people had made. As I looked through them, I was paying special attention to accuracy, which was my number one concern. I also needed to make sure I could correct words entirely with my voice and could also control my mouse by voice.

One piece of software that made it on all or most of the lists was Dragon NaturallySpeaking, it was usually the highest rated on accuracy, had the two other features I was looking for, and it learned from my voice and speech patterns, which was one of those nice-to-have features. I didn’t want to spend hours and hours of my day obsessing over minute details and differences between various products, so I figured I found something that had emerged as a pretty clear winner.

Now, I needed to figure out which version of Dragon to get. I knew that I wanted it on both my work and personal computer, and so the Professional Individual addition let me install it on multiple devices. It would’ve been nice if I could keep the same exact user profile for both (i.e. a cloud-based profile), but that only existed on the Enterprise version (which isn’t available to individuals) so I figured that having a user profile on each device would be fine, both of them would be able to learn from my voice.

Step six: try it out

If it’s software and offers a free trial, do it. If it’s hardware and they have a good return policy, try it out. Online research or people’s recommendations can only go so far. Try it out and see how it works.

I don’t remember if Dragon had a free trial or not, but I do remember the excitement I felt when I was able to directly dictate text with extremely high accuracy. I remember when I was able to use the command “correct previous word” and see a list of corrected words I could choose from. I remember discovering I could dictate punctuation that Google Voice-to-text was unable, such as colon or parentheses. I eventually linked it to my emails and documents so it could learn common words that I used. It wasn’t perfect, there are still phrases that it gets wrong or settings that can’t be changed (such as every time I say “a third,” it will default to 1/3 even if I was trying to say “a third option,” and as far as settings go, either it will always do numbers as numbers or always do them as words, but I can’t tell it to never do fractions but always do non-fractions). Oh well. I knew nothing was going to be perfect. Still, I felt like I had a big chunk of my life back.

So, typing was my major culprit for arm pain, and I had successfully found a way around it. Mouse use was the tougher one.

The main way Dragon lets you control your mouse is via its “mouse grid,” which divides the screen into nine sections, one through three across the top from left to right, then four through six in the middle, and seven through nine across the bottom. Saying a number moves the cursor into the corresponding section and redraws the grid, letting you repeat that until the mouse gets to the place you want, at which point you can tell it to click or double-click or right-click. For example, when I have my Firefox browser open, if I wanted to manually click the button to reload the page, which is in the upper left of the browser, I would say “MouseGrid one,” and once the grid was redrawn in the upper left, I’d see that saying “one” again would get my cursor into the right place. Some elements of the page can be directly accessed by voice, such as “go to address bar,” and good websites are accessible and most of the buttons are labeled, so I could say “click sign in” to click the button whose text reads Sign In, but most websites aren’t very good at accessibility.

It’s fairly clunky and I’m an impatient person. I found myself getting frustrated with the mouse grid and just picked up my mouse and used it, which then would flare my hands and arms and I’d myself up over my lack of willpower or patience. That brings us to the next step…

Step seven: refine and tweak

Maybe your free trial of the software showed you that this wasn’t a good fit for you. Just return it or cancel your subscription whatever you need to do, and then you can go back to previous steps where you outline new features you now know that you need and try something else out.

Or, you discover that it does work, but not 100%. You might need to set up some features, or you might need something else to supplement it. I ended up doing both.

First, I tried a third-party add-on to Dragon called KnowBrainer, which adds new features and voice commands to Dragon. They offered a 30-day trial and pretty much the only feature I liked was their “display numbers,” which assigns every clickable item on the screen a number and then I can directly tell it “double-click 30” and it will double-click the item numbered 30. It worked fine, but it was somewhat slow in assigning all of the numbers, and I still am an impatient person and wanted it to go faster. At the end of the trial, I decided to not purchase the full version. If there were more features I liked in it, perhaps I would’ve bought it.

Since clicking items on the screen was a problem, I asked my hand therapist from mouse recommendations, and she sent me a link to the Evoluent Vertical Mouse, and the feature that grabbed my eye was an auto-click feature you can enable, meaning if you leave your mouse stationary for half a second (or whatever timing you want), it will automatically single click whatever is below it. I couldn’t find any other vertical mice that have auto-click, nor could I find any software where I could add auto-click to another mouse, which was frustrating because this mouse was fairly expensive. But if it works, I’ll take it. Since clicking (not moving the mouse) was my main issue, this worked out perfectly.

Return to step one

At this point in the story, I’ve successfully solved the issues I set out to solve: I type by voice, I can control my mouse by voice but when I get impatient with it, I can use my vertical mouse with auto click.

If you have a chronic illness, you know this probably won’t be the end of the story forever. Perhaps the adaptation allows you to heal and you can return to your previous ways of doing things. Perhaps your health changes and you need to come up with a new adaptation for that, which might not mesh with your existing ones. Regardless, you can continue to monitor the situation and see what you need.

For me, the biggest change was when sitting became too painful, and standing had already been painful, so it meant I needed to lay down. I got a laying desk, and thankfully I already was used to voice typing (reaching my hands up to type at a keyboard would’ve been bad, even if my arms hadn’t been hurting). The problem was I could no longer use my vertical mouse and had to rely entirely on the mouse grid…

Except I recently figured out that Dragon also allows for custom voice commands. If I was regularly having to say “MouseGrid 7 5 6 4 click” to navigate my cursor to the right place to scroll down on the folders list in Gmail, and if I have to do that multiple times per day, I might as well create a custom command called “Gmail scroll down.” Over the course of the week, if I saw myself repeating a command multiple times, I decided to program it in. It’s been amazing and so much faster, and since I like when things go faster, I’ve been less stressed and frustrated at my computer.

How did I miss this before? I wondered to myself. It’s because I didn’t need it before and I didn’t care about it before. It wasn’t a feature I looked into. All I wanted was something that could voice type, correct by voice, and control my mouse by voice. All other features were a bonus. But, as my condition changed, custom commands moved from “bonus” to “essential.”

And had I discovered that Dragon no longer served my needs and I needed to switch to something else, that’s fine. Tools are supposed to serve you, not the other way around. I’m not going to get so bought into a tool that I keep using it if it doesn’t do what I need it to do.

What about you?

I hope my experience and thought process can be a case study to help you find something you need to give you more control, more function, or better quality of life.

If your symptoms are similar to mine, your needs in a tool might be similar to mine and you might find Dragon or another voice software helpful. Or, maybe you have some similar symptoms but there’s a big enough difference that your needs are different and therefore a different tool would do a better job for you. Or, you might have an entirely different set of symptoms and needs, but you can still use the same thought process to figure out what would be helpful for you.

Like I mentioned in the beginning, when I was doing this for myself, I didn’t have a series of steps to follow. But, steps are easier to remember, and these steps accurately capture the process I used in the thoughts that I found helpful. I hope you find them helpful too.