return to table of content

Mouth-based touchpad enables people living with paralysis to use computers

nashadelic
16 replies
1d21h

How did we not have this 10-15 years ago? It just feels like not enough people work on, or care enough about assistive tech till it impacts them personally.

cortensinger
2 replies
1d21h

Exactly. Its unfortunate that this is the way the world works...but we hope that the MouthPad^ can change that tired script! Universal design benefits us all, not just those at the extremes -- we should all care. See the history of curb cuts on our sidewalks as a small example.

susiecambria
0 replies
1d18h

For a short time in DC (District of Columbia, USA) government, I worked with Age-Friendly DC. They were all about universal design, obviously. I learned a ton - like how curb cuts are good for folks using canes, walkers, and wheelchairs, and those who push things like strollers. However, the Age-Friendly DC staff could have done much more to work with non-aged policy advocates and organizations, to show them the benefits of things like curb cuts.

Personally, I'm using a lot of what I learned from the Age-Friendly folks; my 83yo mom has moved in and the learning curve, while steep, got a head start with my government experience.

fragmede
0 replies
1d20h

As someone who's not a quadriplegic, I want one of these.

brudgers
2 replies
1d17h

I went to grad school for architecture at the end of the last century.

Tim was a classmate. He was and remains paralyzed from the neck down. At the computer Tim manipulated a trackball with a tool — rod connected to a flat mouth piece.

This is different, but I am not sure it is better. The trackball and stick were mostly mechanical, reliable, and straightforward to repair.

No special software. No need to charge. Tim’s tools were as reliable as a mouse. Not much of a choking hazard because of the stick either.

I’m not saying the mouth-pad is a toaster fridge. Only that it has technical complexity rather than mechanical simplicity.

eezing
1 replies
1d14h

I would think fewer moving parts is less complex. Considering it’s basically a trackpad, I doubt the hardware and software are bleeding edge.

brudgers
0 replies
1d12h

It’s a radio and computer that goes inside a person’s mouth. That person can’t use their hands. How do they take it out and put it in?

Tim had a red solo cup size container next to his keyboard — he also typed with the tool in his mouth. [1] When he wanted to talk to someone, he stuck the stick in the cup and stood the tool up where he could pick it up again.

[1] I hadn’t remembered that because it’s been 25 years. Tim could also out sketch most people with a pencil in his mouth as well. He had a bachelor in fine art before the motorcycle wreck.

_spduchamp
2 replies
1d19h

I work in a lab at a university where I'm currently working on building assistive music controls for disabled musicians. From what I've seen in my short time here is it seems like a lot of these kinds of projects don't often translate from the research space into viable businesses. Also, due to the highly variable physical and neurological characteristics, often assistive devices will need specific customization, tuning, and support. So these things also tend to be expensive.

stevenbedrick
1 replies
1d19h

This is exactly correct (I worked in AAC and BCI for years, and still collaborate with the group that I used to work closely with). To elaborate a little bit on your excelelnt answer: In addition to being enormously expensive and requiring heavy customization and personalization, assistive technology has other issues that go well beyond the core technical functionality. It also often requires significant up-front investment in configuration and training for the user, which needs to be done by a professional in the field, and then --- crucially --- also requires ongoing maintenance and re-training by skilled practitioners, which means there are ongoing operational expenses.

One of the reasons for this last bit is that many users of assistive technologies have very dynamic situations. For example, if they have a progressive condition like ALS, their mobility and other capabilities and needs will change over time; even if the user doesn't have a progressive condition, their living situation very likely will change over time, which may involve needing to re-work their setup. This might involve something as major as moving from one care facility to another, or having to adjust to the presence of a new pump or something that causes EM interference with your device.

Even for somebody with a very static medical condition and very stable living situation, their computing setup won't be static, so will need ongoing maintenance; or, alternatively, their computing setup will become static if they end up dependent on a particular piece of assistive technology, which can lead to a whole other set of issues (which is how you get people running ancient versions of Windows for years and years and years, because it's what their eye-tracker works on, and the vendor is long out of business and so never got it working on Windows 10).

The other piece to the puzzle is that, speaking as a gross generalization, users of assistive technologies tend not to be wealthy people (many speech and motor disabilities make it challenging to work). So, you've got an expensive product, often involving ongoing operational costs, with zero economy of scale, for a small and relatively poor customer base; this all adds up to what may fairly be described as "a very challenging business". Adding another layer of complexity, the economics of commercial assistive tech in the US have a lot to do with insurance Medicaid rules; the only reason there is any commercial activity at all in assistive technology and AAC is because under some circumstances Medicaid will pay for a device, within certain very rigid and complicated boundaries. So now your business also is going to have to juggle that problem in order to keep the lights on, which will add another layer of constraint to what is already a complicated situation.

On the other hand, when the right technology is paired with the right user and the right support, the impact on the quality of life for the user and their family can be truly enormous. Which is what keeps people working in the space...

ansible
0 replies
1d18h

At least one company I know of (though this was 15 years ago) had ridiculously high prices for their products. But they had extensive after-sales support, and the warranty was such that you could basically run over the unit with a car, and they'd still repair / replace it under warranty for free.

spiritplumber
1 replies
1d21h

we did, there were at least 3 kickstarters (of which i ran one) but not enough people were interested.

fragmede
0 replies
1d20h

that's so cool that you tried. how much did you spend on advertising for your Kickstarter? I totally would have kicked in if I'd heard about it.

oliverx0
1 replies
1d12h

We tried about 9 years ago :) we even open sourced the solution. At the time we felt there was not a big enough market so we decided to make it open for anyone to build upon our research.

http://pallette.io/instructions.html

DoingIsLearning
0 replies
1d9h

Seems to be the case for so many assistive technologies. It is even more frustrating considering how high impact some of these technologies are.

Do you have any insights on how this space could grow other than State Funding or Insurance policy changes?

jagged-chisel
0 replies
1d21h

I suspect it’s a couple things. First, the skill gap - maybe someone begins needing assistive tech and has no idea how to start making. Second, perspective - as an able-bodied techy, I lack the understanding of the problem to build the right thing. Third, communication - can the one in need describe their need? This really requires reflection, and maybe skill. And lastly: money - how many of is can afford to work for months or years on a project that doesn’t (and likely won’t) pay?

itissid
0 replies
1d16h

MITs MO is to try to create a breakthrough tech and rapidly spin off startups from the tech. But I think the main barrier is that to operationalize hardware requires massive capital investment upfront for testing and then regulatory approval, this is mostly not possible for VC. So the students/profs who make the breakthrough do the next best thing: Patent and then sell it to a company that has the capital to do it.

But someone can correct me if I am wrong about this.

trailbits
13 replies
1d20h

I can imagine this helping anyone that has struggled with computer-related overuse injuries. I had a bad case of tendinitis that made using a mouse or trackpad very painful. I would have loved to have this kind of device as another option.

moffkalast
12 replies
1d19h

I'm really wondering why nobody's selling a foot operated mouse yet, it would be pretty practical even for normal people when typing with both hands.

trailbits
4 replies
1d19h

I did experiment with using a trackball taped to the floor. It's hard to get precise positioning moving your foot. I think using the tongue along the roof of your mouth would be more precise and less fatiguing. What was more useful was a set of 9 foot switches that could be programmed to send arbitrary keystrokes. I could off load from my hands the most common keypresses I do all day, for example: pageup, pagedown, tab, enter, backspace, mouse click, passwords... along with speech recognition software you can get at least non-coding tasks done without using your hands too much.

prmoustache
1 replies
1d7h

It's hard to get precise positioning moving your foot.

How long did you do your experiment? Obviously most of us are not typically used to point and move precisely with our feet. So to be efficient we would need to train for months but that doesn't mean we can't be precise with our feet if we were to dedicate enough time for it.

I once saw a documentary about a german woman who was born without arms. She would do everything with her feet, writing, drawing, smoking a cigarette or mounting a horse and holding the reins with one foot. We are much more adaptive than we think we are but we don't realize it when we have the luxury to not needing it.

RogerL
0 replies
1d2h

I went to grad school with a guy w/o arms. Sure, he could write with his feet. It was extraordinarily hard and difficult. Nothing changes the physical facts - we neither have the nerves or fine control in our legs/feet that are present in our hands.

We also build fine motor control in very young childhood; children can't color within the lines (for example) because the nerves just aren't there yet. I would expect someone born without arms to have better fine motor control of their legs than an adult that tries to develop the ability. And it takes them years - 7 or 8 years to get pretty decent control. Even if I had the neuroplasticity of a young child (I surely don't) I wouldn't want to spend years trying to develop that control.

I face this every day on the piano. You distribute motion between fingers, wrist, forearm, elbow, and shoulder based on the amount of fine control needed. Sure, you can use your shoulder to trill, but it is always going to be very slow and clumsy. finges are best, but you face fatigue, so you usually use wrist rotation to get muscles with more endurance suppling the gross motions, and then fingers for the fine control of dynamics. You can't change this physical reality, at the most you can compensate (e.g. someone with a fused wrist would have to find alternatives).

fallingsquirrel
1 replies
1d18h

Can you link the hardware you're using? I have one foot pedal for push-to-talk and I think more could be useful, but I don't want to tie up 9 USB ports.

trailbits
0 replies
1d16h

Yes, push to toggle microphone is another great use for these! I was using the Kinesis Programmable Foot Switch 26 years ago with ps/2 connectors (later they switched to USB). You would get 3 foot switches in each unit and chain them together along with your keyboard so they only used one host port. The most similar thing I see now would be https://www.amazon.com/X-keys-Foot-Pedal-Playback-Control/dp... It's frustrating that I can't reprogram my old foot switches since the manufacturer stopped supporting them. If buying today, I'd look for something based on open source QMK firmware like my keyboard uses. This project looks cool: https://github.com/christrotter/qmk_firmware/tree/arcboard-s...

hinkley
4 replies
1d19h

Best I personally managed was push-to-talk via a foot pedal, to help with mic discipline in cooperative games without reducing my reaction time.

I'm not sure my feet have the accuracy to work as a mouse.

I think assistive devices do better as a class when some fraction of them are also usable or even attractive to able people as well. It gets us thinking about other modes of interaction and of course it brings the unit price for those items down out of the stratosphere.

Foot mouse will unfortunately stay niche, but when I saw the device in this article, I thought, "Why not for everybody?"

Levitating
3 replies
1d9h

I'm not sure my feet have the accuracy to work as a mouse.

You should be thinking more of like a trackpoint for your feet/foot.

Foot mouse will unfortunately stay niche, but when I saw the device in this article, I thought, "Why not for everybody?"

I can feel my tongue having a muscle ache just by looking at this.

moffkalast
1 replies
1d9h

I don't get the tongue idea either, any kind of device you'd put in would make it hard to talk and who wants to eat a battery? Plus the movement area is pretty tiny.

Accurate foot movements are just training I bet, just like with anything else. Arm amputees have shown again and again that it's just a matter of practice. Plus, were any of us really that proficient with a mouse when we first started? We take it for granted but we've been honing that for decades.

trackpoint

Looking at how that one works, it's just four load cells. That would be exceedingly simple to reproduce with a microcontroller that supports HID. I might actually try to make one.

pragma_x
0 replies
1d4h

FWIW, I've considered mapping a midi pedal board to meta keys and macros before. That alone could be useful - you'd go from a typist to something like an organist.

RogerL
0 replies
1d

Lift your foot up and try to balance in your seat and see how it feels. Try using a standing desk with it. Put a pen between your toes and try to draw a square and feel major muscles in your upper thighs struggling to make these fine control motions. Put on high heels and try to do anything that isn't gross movement. Sit in a train/plane/car and think of trying to use a laptop with a foot controller. Heck, put your mouse on the ground right now, turn pointer speed down as slow as possible to accommodate the gross movements of your legs, and try to use it.

xnx
10 replies
1d22h

Assistive technology for the disabled is the higher use for mouth-input, but it'd be great to have a simple bluetooth mouthguard with media playback controls (play, pause, next previous) for action sports.

sa-code
8 replies
1d21h

Could you provide some examples of this?

xnx
7 replies
1d21h

While snowboarding or skiing, it is difficult to control music playback because bulky gloves are being worn. Cycling might be another sport where you want to control music without taking your hands off the handlebars. A single button (like tapping an earbud) that could be gently bit down on could control song playback. Prior art for this type of device would be something like this: https://www.airturn.com/products/airturn-bite-switch-remote-...

rqtwteye
5 replies
1d21h

It's very strange to me to listen to music while snowboarding or cycling. Especially while cycling you need a lot of situational awareness of what's going on around you. I think it's very dangerous not to be able to hear cars coming up from behind.

fragmede
0 replies
1d20h

in other parts of the world there are dedicated bike trails in nature where there is no danger from cars

el_benhameen
0 replies
1d13h

Some headphones (eg airpods) have pass-through audio to allow situational awareness. There are also trails in my area that allow bikes but that are very remote and are unlikely to have much bike and pedestrian traffic. It wouldn’t be unsafe to have headphones in there.

callalex
0 replies
1d16h

You’re absolutely right, but a truly startling amount of people on the trails give absolutely zero care about other trail users and are just reckless antisocial parasites.

Sohcahtoa82
0 replies
1d1h

Even without the need for situational awareness, it feels weird to me to feel the need to constantly hear music.

When I do a bike ride, for me, the silence is a feature.

Ichthypresbyter
0 replies
1d20h

Replying to the reply- even in dedicated car-free recreation areas, there are other people doing the same thing as you but going faster.

fragmede
0 replies
1d20h

voice control is pretty convenient for those environments

imiric
0 replies
1d9h

Yup, I can see it now:

- Single tap on the roof of your mouth: play/pause.

- Tap on the inside of your upper left/right teeth: previous/next.

- Drag your tongue along the inside of your teeth: playback scroll.

- Drag your tongue on the roof of your mouth from front to back and back to front: volume up/down.

It seems very intuitive, and like it would be much more precise than any of the finger gestures we use today. My only concerns would be comfort, mistaken inputs, and impact on speech. But then again, some people wear dental jewelry and it doesn't seem so bad. :)

mrmetanoia
4 replies
1d16h

I just want this for extra control options in gaming. I bet I could get good at aiming with my tongue. Shorter distance brain to tongue than brain to hands too? Faster response time? Tongue control is going to dominate in esports ;) RGB mouth guards incoming.

kaashif
1 replies
1d15h

What is the average gamer's tongue stamina or tongue dexterity like? How long do you have to practice to improve for this to actually be useful?

I feel like it's probably pretty hard to get good with this.

RoyalHenOil
0 replies
1d14h

Some people have much stronger and more nimble tongues than others due to physical differences (e.g., tongue size, the length of the tongue frenulum, etc.). But this is the case for the hands as well, which is why people have different input preferences and different gameplay preferences.

Personally, I think I would prefer a mouth controller over a traditional controller. I am very prone to tendinitis in my hands, plus I have small hands and short fingers, and can't play twitchy games well because of it.

eimrine
0 replies
1d11h

The picture of a tongue mouse from article (maybe it's a render I don't know) seems to have maybe 16 dpi so it is probably fits for secondary mouse just for rotating the world, but for tiny pixel-hunting I would like to have something with a multiplicator.

Geenirvana
0 replies
1d5h

There is a streamer named RockyNoHands[0] He plays Warzone entirely with his mouth. His controller has tubes that he blows and sucks to control the character.

It is really impressive watching him live on such a competitive game.

[0] https://en.wikipedia.org/wiki/RockyNoHands

kolbe
3 replies
1d2h

I always thought this would be the best way for anyone to operate a mouse. Not just the disabled.

freedomben
2 replies
1d2h

Better than eye-tracking? It's all though-experiment land for me, but when I think about the precision and speed with which I can move and "click" something, nothing beats the eyes.

kolbe
1 replies
22h53m

Maybe. I just think of all the other things that my eyes want to be doing. I figure that would interfere too much in the usefulness. But i agree. We should just test out all the wacky ideas and see what works best.

freedomben
0 replies
57m

Great point, maybe that is the weakness in eye tracking. What about a small switch in the mouth you press with your talk, similar to a push-to-talk that activates the eye tracking. When you aren't pressing, you can look around and do other things with your eyes without affecting the computer?

Wish I had the bandwidth to take this on right now, because it sounds super fun and something I could be very passionate about.

vinceguidry
2 replies
1d20h

Aren't there already products that use eye tracking to locate pointer clicks?

mrguyorama
0 replies
21h36m

Yes, Tobii sells cheap cameras and SDKs that have done exactly this for years now, to the point that several random games have Eye Tracking integration for no reason.

avtar
0 replies
1d17h

Something like Eye Gaze perhaps

vasco
2 replies
1d20h

I would use this before any voice assistants our neurolinks.

tomasero
1 replies
1d20h

We're adding silent speech input and bone conduction audio feedback to the next version of the MouthPad^.

causality0
0 replies
1d14h

Also consider perforations in the inert parts of the MouthPad. Especially considering its intended users, it's a choking hazard and sooner or later somebody will get one of these stuck in their throat. You'd rather it not kill them when that happens.

titzer
2 replies
1d3h

Forget code smells. "This code tastes bad."

zelphirkalt
1 replies
23h21m

This was written by someone with bad taste.

th3w3bmast3r
0 replies
23h3m

This definitely left a sour taste in my mouth.

amanzi
2 replies
1d21h

Fascinating. I started to instinctively "scroll" with my tongue while reading the article - felt really intuitive.

flemhans
0 replies
1d21h

Tickles a bit. But the pad might help.

ZuLuuuuuu
0 replies
1d7h

Exactly! I also started to pretend scroll with my tongue and it actually felt quite intuitive. It is a very clever solution! But they will probably need a well written driver which rejects accidental touches since at any point in time my tongue touches the roof of my mouth.

Zenzero
2 replies
1d22h

What an incredible piece of technology. I'm very impressed.

tomasero
0 replies
1d20h

Thank you! We're just getting started :)

Zenzero
0 replies
1d18h

Why am I being downvoted for saying this?

People on HN complain how a substantial amount of the comments are negative. I give my honest opinion, as a doctor, knowing full well how hard it is to not only clear the hurdles to get to where they are at, but I also appreciate that it is a worthwhile endeavor. In an era where half of the people in tech are wasting their lives optimizing SEO and to sell more ads for useless crap, these people are out there making tangible improvements in people's lives.

LoganDark
2 replies
1d20h

Oh hey, I remember this, signed up to beta test a year or so ago and 6 months later got an email that said I was rejected for being outside of California. I think at the time I found it weird that it took them 6 months.

tomasero
1 replies
1d20h

Hi Logan, thanks for your comment, and apologies for the delay in our response. We're a small company with limited resources, so we've prioritized waitlist applications from those who really need our solution and are close to us. We're working on scaling our operations and hope to serve everyone in the US soon. Stay tuned!

LoganDark
0 replies
1d6h

It's fine. I'm not where the money and news coverage is. Go help those quadriplegics.

thih9
1 replies
1d4h

The recently launched Vision Pro and its eye tracking also offers a relatively niche and very accessible input method. It is already being used in this context:

“I was having trouble getting around that day,” he said, so he took the day off and did work for his consulting agency, Equal Accessibility, from bed, wearing the Vision Pro, surrounded by screens he controlled with his eyes and a series of custom mouth-sounds that triggered selections. “As I get older, and this happens more often for me,” he said, “I envision myself working virtually with the AVP even more.”

https://nymag.com/intelligencer/article/apple-vision-pro-dis...

wlesieutre
0 replies
1d4h

Eye tracking is coming to iPhone and iPad too. I’m sure it won’t work quite as well as the Vision Pro’s hardware, but for someone who can’t take a headset on and off by themselves it’s nice to have options.

https://9to5mac.com/2024/06/13/turn-on-iphone-eye-tracking-i...

Sohcahtoa82
1 replies
1d1h

People living with such extreme disabilities are so much stronger than me.

I put a lot of pride in my independence. If I needed someone else to feed, clothe, and bathe me, I think I'd rather die. I'd feel like I was no longer living, but merely surviving.

zelphirkalt
0 replies
23h17m

Probably most people would think like that, but it all depends on what you can still do. If you become a good writer or even if you stick to software, but make valuable contributions, that improve the world a bit, or become a great orator, or give people good counsel, or any other great activity, you might find, that that is a goal worth still continuing to live.

th3w3bmast3r
0 replies
23h3m

Really cool tech!

pragma_x
0 replies
1d4h

Does anyone here know how the sensor technology actually works? Is it capacitive touch, a low voltage electrical connection, or something else? Also, how would something like this differentiate mouse use from speech?

nsonha
0 replies
1d16h

I think in the early Iron-man comic books, it's said that he had some switches controlled by his tongue in the helmet

mongol
0 replies
1d20h

Is there a site where such tools are listed? Some of these may help people without disability too.

geepytee
0 replies
1d20h

Interesting that he built this after getting closely familiar with Neuralink

freedomben
0 replies
1d2h

Kind of tangential, but I don't understand why keyboard/mouse layers haven't caught on more. I kickstarted the UHK[1] years ago, and haven't had a mouse ever since. It does take a little bit of getting used to, but controlling the mouse with a keyboard is really amazing.

[1] Ultimate Hacking Keyboard

cromniomancer
0 replies
1d20h

Nice to get this before The Jackpot arrives!

causality0
0 replies
1d3h

Can we talk about the absolute perfection of the name? MouthPad sounds like someone trying to say mouse pad with a MouthPad in their mouth. It's beautiful.

brezelgoring
0 replies
1d20h

Not exactly related to disabled people, but a though crossed my mind on how cluttered F1 pilot wheels are with buttons, I imagined some F1 Engineer already thinking on how feature dense these pads can get and how to integrate them into races.

Not related, I know.

b3lik3wat3r
0 replies
1d21h

wholesome content :') x.com/augmentaltech/status/1798701297748902349

Gloomily3819
0 replies
1d7h

Sounds like a pretty good solution for those who can't wait for neural interfaces to become available (and affordable).

Fengjiao
0 replies
1d21h

where are you building them? How much does it cost to build?