return to table of content

Show HN: Open-source digital stylus with six degrees of freedom

bloopernova
42 replies
19h29m

Very cool. They've done what I've daydreamed about, and actually got it to work!

When I playedElite: DangerousI used a "hand on throttle and stick" (HOTAS) setup, along with foot pedals. I couldn't help but think that there must be a better way to control a spaceship: your ship can pitch, yaw, and roll in addition to being able to fire thrusters in 6 directions.

I wanted a handheld ship model that I could move such that the ship inElitewould move in the same way. The linked project looks like it could do just that. Thrust would be controlled in a similar way, but with my other hand.

Strange or new input models like that are so amazing to me. Our imagination can really fly high with these sorts of capabilities.

navane
15 replies
18h56m

this 6dof "joystick" has been around for decades:https://3dconnexion.com/nl/spacemouse/

marcosdumay
7 replies
17h7m

I've never found an explanation of what exactly is in that.

Is it a track-ball, a rotary encoder, and a 3D stick?

Doxin
3 replies
9h47m

Imagine a 3-axis joystick that can also sense translation. So aside from pitch/roll/yaw you also get the linear up-down/left-rigth/forward-back axises.

imsaw
2 replies
7h29m

I can see how pitch roll yaw would work, but I can't imagine how linear movements are made on the same joystick? Maybe up and down is push and pull but what about the other directions? Do you push the device itself?

taeric
0 replies
2h2m

The trick in comparing it to a joystick is that it can distinguish between tilting the control to a side, and pushing the control to the side. Does that help? Similarly, it can detect pushing straight down and pulling straight up.

numpad0
0 replies
4h8m

Looks like current incarnation is Stewart platform lookalike optical setup for all axes[1]. I misspoke in a different comment: Mine was older SpaceBall 5000 model, and it was more like two joysticks joined at the stem.

1:https://www.fictiv.com/teardowns/spacenavigator-3d-mouse-tea...

numpad0
0 replies
9h10m

It’s a knob that spring back to its origin when let go, that can move ~45 degrees or ~1/4” each axes(subjectively). It feels a bit like those old bobbing dashboard toy in hand.

Mechanically, it’s something like an Rx/Rz two-axis joystick with sliding knob part on top that can be slid in X/Y, pushed/pulled in Z and twisted in Ry as well. All axes are spring centered.

lambda
0 replies
16h23m

There's a picture on the Wikipedia article:https://en.wikipedia.org/wiki/3Dconnexionand there's a patent you can look at:https://patents.google.com/patent/US20050172711A1/en?oq=2005...

It's flexible, and there are LEDs and linear optical sensors set up with an occluder in between that casts a shadow (or passes a light through a slit, according to the patent). The sensor detect where along the sensor the shadow or light falls, which indicates how much you have translated or rotated the control in a particular axis.

abe_m
0 replies
16h42m

I don't know exactly, but using one, it has the sensation of using the trackpoint on a Thinkpad. The puck you push, pull, and twist to make the motion is a bit springiness. The harder to push it, the faster the motion happens in the software on screen. When you release, it springs back to neutral. The old ones (late 90's vintage) had trouble coming back to center, and there was a hot key to rezero it if it was starting to drift. The newer ones don't seem to have the issue.

two_handfuls
4 replies
17h30m

This. Sadly the drivers suck.

Isthatablackgsd
1 replies
17h2m

Would it help with scripting macro software such as AutoHotKey? I imagine it wouldn't pick up some inputs due to proprietary driver.

It have the same issue for my Logitech G600 that it can't pick up some inputs. I set up the macros in LGHUB with keystrokes (multiple modifiers with keys) and use AHK to grab those and expands the capabilities of my G600 than what LGHUB offers.

two_handfuls
0 replies
14h59m

Exactly, the inputs aren’t exposed as extra axes so those tools don’t see them.

radarsat1
0 replies
7h53m

I've always found it fascinating that companies that makes specialized hardware devices like this lean so hard into proprietarysoftware, as if that's the thing that gives them the edge on the competition.

If anything the hardware is the hard part to copy and open source drivers could do nothing but benefit them and ensure wide platform compatibility and longevity, and yet so many companies like this insist on closing things up and forcing hackers to reverse engineer everything. It's bizarre logic to me.

abe_m
0 replies
16h40m

There is an opensource driver package for Unix type systems[1]. I find them to be pretty reliable. The stock Windows drivers do leave something to be desired.

[1]https://spacenav.sourceforge.net/

oceanghost
0 replies
13h10m

Yup. Use one every day for Fusion360 and Blender.

RF_Savage
0 replies
16h21m

And it works surprisingly well for Descent. The original 3dconnexion one worked first try on linux too.

userbinator
4 replies
17h42m

Given that a spaceship isn't all that different from a fixed-wing aircraft, and the stick + pedals arrangement has become a standard for controlling the latter for over a century, I think it'd be hard to improve on.

lowbloodsugar
1 replies
15h50m

Do fixed wing aircraft spend a lot of the time going backwards? Do they, for example, need to turn around and point their engine in the direction of travel in order to slow down? If a fixed wing accelerates to 1000mph and then rotates it’s nose by 90 degrees, how fast is it going and in what direction? What about a space craft?

nkrisc
0 replies
15h23m

If a fixed wing accelerates to 1000mph and then rotates it’s nose by 90 degrees, how fast is it going and in what direction?

I don't know how fast, but depending on which axis it rotates on and how fast, it may be going in several directions at once.

marcellus23
0 replies
16h35m

The difference is that spaceships have thrusters, so they have the ability to accelerate up, down, left, or right, in addition to all the controls of a fixed-wing aircraft (throttle + stick + pedals).

hooverd
0 replies
16h12m

With no gravity or drag they're way different.

Arelius
4 replies
19h14m

I still hold onto my SpaceOrb369 for this sort of game. There is a guy that makes a converter to make it work over USB...

https://en.m.wikipedia.org/wiki/SpaceOrb_360

https://www.tindie.com/products/vputz/orbotron-9001-version-...

myself248
2 replies
18h38m

I have the slightly newer version, branded by HP as the SpacePilot, and use it daily in Fusion 360 CAD. It has a native USB connection, and 6 user-definable hotkeys below an unimpressive LCD.

It requires some old drivers that aren't officially supported (why would they remove support for perfectly good hardware from the newer drivers? To send good stuff to the landfill, of course!), but when Autodesk tried to move Fusion to the new driver model exclusively, user outcry persuaded them to leave the old drivers in as an option. Apparently there's quite a few of us using those SpacePilots, and the phrase "from my cold, dead fingers" comes up not infrequently.

w-ll
0 replies
16h9m

I plugged in my old SpacePilot Pro the other day and was so dissapointed that the drivers where so outdated, and like nothing was working.

Arelius
0 replies
16h25m

For sure, the HP one is a rebrand of the 3Dconnexion right?

Those are great, but nothing really compares to the original SpaceOrb360 controller form-factor for gaming IMO.

lowbloodsugar
0 replies
15h56m

If you weren’t playing Descent with this you were losing.

matheusmoreira
3 replies
18h6m

I couldn't help but think that there must be a better way to control a spaceship

Two sticks?

https://old.reddit.com/r/EliteDangerous/comments/16xi20a/dua...

https://youtu.be/T2-IHgNYaKA

bloopernova
1 replies
16h45m

That's really nice, before now I hadn't really understood how to use 2 sticks, but now it makes sense.

I used a CH Pro throttle, which has enough hats to sort of mimic that 2 stick action. Main throttle back forward for main engine and retros. A smaller thumb stick/hat for each of Y and Z axis.

https://www.chproducts.com/Pro-Throttle-v13-d-719.html

sundvor
0 replies
14h27m

That works wonderfully with VKB's STECS, which lets you swap physical detent profiles in roughly 20 seconds. So you have a forward/backward axis on your throttle with a well defined "W" centre point, plus a high quality thumb stick for your lateral/vertical thrusters.

(In case anyone is wondering, I transitioned from the Warthog throttle with DeltaSim's slew mod with zero issues, I find the STECS Standard way more versatile and flexible).

I couple this with a Gunfighter Ultimate stick (V3), and Crosswinds pedals, all profile mounted. (My GFU grip has twist, but I don't use it.)

When you add TrackIR in my case (or VR for others), there's quite a bit of immersion going on! :-)

I play DCS and Star Citizen these days, also used to fly Elite Dangerous though. Building the combined quad screen workstation / simrig has been a wonderful side project over the past few years. ;)

jessriedel
0 replies
17h10m

Yep. I played Descent II this way and it was glorious.

mulmen
2 replies
16h45m

There’s an entire rabbit hole of dual stick setups for space games. Quack HOSAS to get started.

bloopernova
1 replies
15h55m

Ok Quack is adorkable.

What would be the word for "To Kagi"? They use "Fetch" in the search field, but it hasn't really stuck with me.

master-lincoln
0 replies
5h38m

Why mention the provider instead of using the verb you use the provider for? You want to "search" for something.

You wouldn't say I am AT&Ting or AOLing for browsing the internet or Honda-ing / Ducati-ing for using the motorcycle

sillysaurusx
1 replies
15h12m

One of my fondest memories is when I was 16 or so. I bought a 3D mouse (it’s like a mouse that you can control by waving it around) and taped it onto a hat that said Marines. Then I immediately went into CS: Source and got ready to annihilate everyone.

I think I played about the same, but it would’ve been funnier if I’d gotten creamed.

The theory was that my head is much more precise than my hands, so using a combination of both should improve my aim. It might sound like you’ll end up looking left and trying to see out of the corner of your eye, but in practice the two-mouse system ends up working out. I’ve always wanted to revive the idea for modern times, but you end up looking like a huge dork with a dildo glued to your head. I like that style though.

Your idea sounds way cool. You should make it!

pxndxx
0 replies
6h18m

Modern consoles (and modern controllers I assume) can do this "double-input" for aiming, using a joystick and a gyroscope. I'm still bad at shooters, but it feels super good. Plus you don't look like a huge dork with a dildo glued to your head.

pests
1 replies
18h58m

I always imagined a sphere suspended / held by a minimum number of strings (or rods?) / attachments. By physically pushing, pulling, and twisting the sphere you could detect these movements via compression and tension in the attachments. You could motorize those strings/rods to give resistence and feedback to the piloting.

bloopernova
0 replies
18h52m

Yeah that's almost exactly what I was daydreaming of! Weaker gyroscopes might mean a ship can rotate in one direction slower than others, and feedback to a controller could model that for the user.

tadfisher
0 replies
19h4m

In the sci-fi seriesThe Expanse, theRocinanteis controlled in part by a 3DConnexion SpaceMouse, commonly used in CAD and 3D modeling.

squigz
0 replies
19h20m

This is why 2 sticks is a fairly common setup for space games

blechinger
0 replies
8h44m

The folks at Sublight Dynamics had a really neat product in this vein that I was excited about. Seems like the perfect flightsim controller. They had trouble getting fully to market. Website is still up though so I'm holding out hope!

Terretta
0 replies
1h47m

I couldn't help but think that there must be a better way to control a spaceship: your ship can pitch, yaw, and roll ... in 6 directions.

15 years later and I'm still convinced that the SpaceOrb 360 is THE GREATEST GAME CONTROLER OF ALL TIME. See what I did with the all cap there? It's the G.O.A.T. and it's a tragedy that it lost the battle to "WASD" keyboard and mouse gaming and the "two thumbs" style of controller.

If you've ever tried a SpaceOrb, you probably already know this. If you didn't live through the SpaceOrb's hayday (around 1994-5 with the original Quake, and Descent, truly one of the greatest games ever) then this post will sound like I'm a crotchety old dude trying desperately to relive my the golden hour of my youth. Ok, so you got me. What of it? ;)

https://www.hanselman.com/blog/the-best-controller-for-fps-a...

Degrees of freedom:https://hanselmanblogcontent.azureedge.net/WindowsLiveWriter...

I had purchased a box of these when they were going away, and sponsored a custom driver. For a while there were other solutions available, like OrbShield:

// via SeedStudio:https://www.seeedstudio.com/Orbshield-v1-0-kit-p-671.html?qu...

// via Tindie:https://www.tindie.com/products/vputz/orbotron-9001-version-...

Matumio
0 replies
7h40m

If you like the puzzle of firing thrusters manually, try "ΔV: Rings of Saturn". The game is 2D, but you get six keys to individually fire different thruster combos, and a separate key to fire the main engine.

Then try staying in control with an unbalanced ship, or after you inevitably smashed one of the thrusters into a rock. It's a pretty space-nerd game too, they simulate details like reaction wheel saturation.

charcircuit
9 replies
20h6m

Too bad this wasn't made as an OpenXR API layer so that it could be used with existing software

ipsum2
7 replies
19h38m

The great thing is that its open source, so you can add whatever niche APIs you want to it!

charcircuit
6 replies
17h22m

Why would I be the one to do that? I already have a pressure sensitive 6dof stylus that is compatible with openxr.

OpenXR isn't a niche API when you are dealing with 6dof input devices.

cududa
3 replies
14h22m

I know this might be shocking, but OP didn’t make this for you. There’s actually all sorts of people out there making all sorts of things without considering your specific needs

charcircuit
1 replies
14h9m

Why are you using that tone? I never had the intention to use this. I am just stating that there is a missed opportunity that exists. Why are you implying that I want this as feature for me to use?

jcparkyn
0 replies
12h8m

I'll chime in and say that OpenXR integration is a good suggestion, just not something I had the time/effort to implement (this is a university project, after all, and was focused more on the core tracking tech). I think people just took issue with the way you phrased your original comment.

bloopernova
0 replies
13h7m

Please don't feed the troll. They're being deliberately argumentative.

grodriguez100
1 replies
13h30m

Why would I be the one to do that?

Because you are the one who is complaining.

bloopernova
0 replies
13h6m

Please don't feed the troll. They're picking fights on purpose.

noddingham
0 replies
19h41m

It's open source so...

crazygringo
8 replies
18h16m

Very cool. The use of a webcam really makes me wonder if there's a future where our regular single ~78° FOV webcams are going to be replaced by dual (stereo) fisheye webcams that can:

- Enable all sorts of new UX interactions (gestures with eye tracking)

- Enable all sorts of new peripheral interactions (stylus like this, but also things like a steering wheel for racing games)

- Enable 3D 180° filming for far more flexible webcam meetings, including VR presence, etc.

The idea of being able to use the entire 3D space in front of your computer display as an input method feels like it's coming, and using a webcam the way OP describes feels like it's a little step in that direction.

swanee
2 replies
14h25m

I thought this was around the corner years ago when Intel and partners had RealSense modules being built into laptops but it seems like all the players have shifted focus to more enterprise and industrial markets.

westurner
1 replies
13h14m

Wii Remote (2006), Wiimote Whiteboard (2007), Kinect (2010), Leap Motion (2010- Ultraleap (2019)),

There are infrared depth cameras in various phones and laptop cameras now.

[VR] Motion controllers:https://en.wikipedia.org/wiki/Motion_controller#Gaming

Inertial navigation system:https://en.wikipedia.org/wiki/Inertial_navigation_system

Inertial measurement unit :https://en.wikipedia.org/wiki/Inertial_measurement_unit:

An inertial measurement unit (IMU) is an electronic device that measures and reports a body's specific force, angular rate, and sometimes the orientation of the body, using a combination of accelerometers, gyroscopes, and sometimes magnetometers. When the magnetometer is included, IMUs are referred to as IMMUs.[1]

Moasure does displacement estimation with inertial measurement (in a mobile app w/ just accelerometer or also compass sensor data?) IIUC:https://www.moasure.com/

/? wireless gesture recognition RSSI:https://scholar.google.com/scholar?q=wireless+gesture+recogn...

/? wireless gesture recognition RSSI site:github.com :https://www.google.com/search?q=wireless+gesture+recognition...

Awesome-WiFi-CSI-Sensing > Indoor Localization:https://github.com/Marsrocky/Awesome-WiFi-CSI-Sensing#indoor...

3D Scanning > Technology, Applications:https://en.wikipedia.org/wiki/3D_scanning#Technology

Are there a limited set of possible-path-corresponding diffraction patterns that NIRS (Near-Infrared Spectroscopy) could sense and process to make e.g. a magic pencil with pressure sensitivity, too?

/q.hnlog "quantum navigation":https://news.ycombinator.com/item?id=36222625#36250019:

Quantum navigation maps such signal sources such that inexpensive sensors can achieve something like inertial navigation FWIU?

Fromhttps://news.ycombinator.com/context?id=36249897:

Can low-cost lasers and Rdyberg atoms e.g. Rydberg Technology solve for [space-based] matter-wave interferometry?[...]Does a fishing lure bobber on the water produce gravitational waves as part of the n-body gravitational wave fluid field, and how separable are the source wave components with e.g. Quantum Fourier Transform/or and other methods?

Because the digitizer

westurner
0 replies
12h59m

e.g. a magic pencil with pressure sensitivity, too?

Wouldn't such a capability also be useful for surgical AR/AI, robotics, and training?

moritonal
2 replies
17h38m

I'm sadly bearish on this kind of stuff. The Mac touch-panel showed that even a context-aware, reactive and fully integrated input struggled to break the flow gained by keyboard+mouse.

Even today some people are still on the fence about the new fangled mouse!

two_handfuls
1 replies
17h28m

The Mac touch bar sucked, that why it didn’t gain traction.

moritonal
0 replies
40m

Likely valid, but can you say why it sucked? And why any other input wouldn't struggle with the same faults?

mjcohen
0 replies
15h26m

I have an Anker PowerConf 300 webcam which (on Mac or Windows) can go up to 115 degrees with the AnkerWork software.

0x0203
0 replies
5h55m

I'm sure you've heard of TrackIR, which can do head tracking with some head-mounted reflectors. But without mounting anything to your head, the tobii eye tracker can do both head and eye tracking completely passively. It seems like adding hand/finger gesture controls, and peripherals like stylus, mouse, etc... would be an obvious next step, but it doesn't look like they or anyone else is doing that.

tomcam
5 replies
19h36m

This project was part of my undergraduate thesis for electrical engineering. I

Undergrad! If you didn't get top marks on this there is no justice

master-lincoln
1 replies
5h33m

Why do you think so? When I was doing undergrad, the emphasis was on scientific work. So novelty or applicability were not important for grades, but if your written communication about it was scientific. I didn't see a write-up of this project, so I couldn't judge it grade wise

Naracion
0 replies
2h15m

I agree, and this is one of the things that are problematic with academia. Not that scientific rigor / work / dissemination shouldn't be rewarded--it should. But so should work like OP's.

jcparkyn
0 replies
19h23m

Thanks! I'll get results in about two weeks (fingers crossed)

cududa
0 replies
14h26m

I scrolled down to the picture first and the desk made me instantly think “ah, a dorm room”

an_aparallel
0 replies
17h48m

uni marks != quality :-) I agree with you though.

extraduder_ire
5 replies
20h10m

Outside tracking with a camera is not something I would have thought of. Seems cool.

Reminds me of how sad I am nobody's done a good job of cheaply cloning the lighthouse tech that valve/htc use.

jcparkyn
2 replies
19h55m

I should point out that I'm not the first person to use camera tracking for this [1], but to my knowledge there hasn't previously been a serious attempt to combine it with inertial or pressure sensors (which are both necessary for competing with graphics tablets) or make it open-source.

[1]http://media.ee.ntu.edu.tw/research/DodecaPen/

two_handfuls
1 replies
17h23m

Isn’t camera tracking + inertial sensors exactly how the Oculus Rift headset & controller tracking worked?

jcparkyn
0 replies
17h1m

Yes (or something like that, I don't know all the details), but they use infra-red light which requires a dedicated camera. I think they might also require more than one camera (although it is possible to do monocular IR tracking).

mtsr
1 replies
18h51m

AFAIK you can actually get the sensors used for tracking at somewhat OK prices. But iirc the sensors actually do some part of the position calculations and as such aren’t simple enough to be really cheap.

r2_pilot
0 replies
11h58m

Yep, they're actually just tiny solar panels! I got several trying to figure out how to use the lighthouse system. Not enough hours in the day and at the time there wasn't enough compute in a small package cheap enough for me.

ipsum2
4 replies
19h37m

Very cool! It has the added benefit of being able to manipulate objects in 3d. How does it compare to graphic tablets in terms of accuracy?

jcparkyn
3 replies
19h25m

Currently it's not quite at the level of graphics tablets, but it's not too far off, and I think there's quite a bit of potential to improve it using similar techniques [1].

In terms of absolute accuracy, I measured an average error of 0.89mm (for the position of the tip) across the entire area of an A4 page with the camera in one place. In practice you have more precision than that though, because most of the errors are constant biases (not random noise).

For example, here's one of the tests [2] I did for the thesis, which compares the recorded stroke to what I actually wrote (scanned from carbon paper). After aligning the two captures (as a global 2D position offset, everything else is retained), the average distance from the recorded stroke to the scan was 0.158mm.

[1] This paper (which I linked in another comment) uses some more advanced techniques for pose estimation which could definitely be applied here (but it's closed-source, and I didn't have time to re-implement it from scratch):http://media.ee.ntu.edu.tw/research/DodecaPen/

[2]https://github.com/Jcparkyn/dpoint/files/13329235/main-sketc...

orbital-decay
0 replies
12h54m

One thing with using graphic tablets for handwriting and drawing in practice is youreallywant your stylus to be thin, lightweight (15-20g), and have either neutral center of gravity or balanced towards the tip slightly. Also preferably battery-free (non-Wacom tablets were forced to use batteries due to patents, not anymore).

Until it's solved, the better practical use for this is probably a controller for VR or 3D software.

moritonal
0 replies
17h34m

Brings back joyful memories of me begging random Professors from around the world over email for the source code of their papers.

abdullahkhalids
0 replies
16h14m

Nice. It seems there is a systematic error towards the edges of the paper, which can easily be fixed in software.

Though it would be nice to do more tests. For example, there is some error when drawing sharp corners. It would be instructive to see how that changes as you change the angle, etc. And how that changes based on location on the paper.

wnolens
1 replies
14h12m

This is really cool, and right out of my dreams.

Since the move to full remote work at my company, I've been longing for whiteboard sessions. The best I could conjure up was to use an old iPad and load up a shared whiteboarding webapp in a browser on both iPad and my desktop, and then share screen from my desktop, using a cheapo stylus on the iPad to draw.

It's pretty good, but my iPad is super old and slow. Replacing it is not affordable. But a conventional web cam + a relatively low-tech stylus would be much better, and drop the need for an external device completely.

abdullahkhalids
0 replies
13h51m

The cheapest Wacom tablet [1] is 67CAD. Other brands have cheaper ones.

[1]https://wacomstore.ca/product/one-by-wacom-small/

tomp
1 replies
18h27m

Very cool!

Could be useful for robotics / VR as well. One-camera hand tracking anyone?

Question: could you use gyro+accel to track pressure as well? Or at least "taps"?

Another question: how much does it cost? in particular, the pressure sensor...

jcparkyn
0 replies
17h58m

1: You could absolutely use gyro/accel for detecting taps, but for proper pressure sensitivity (i.e., changing pressure in the middle of a stroke), there's not much you can do except have a pressure sensor. It's theoretically possible with a sufficiently accurate pose estimate and a springy pen tip, but not feasible at the level of accuracy I got.

2: I paid about $20 AUD for the pressure sensor, but they can be had for quite a bit cheaper (~$5 USD) in the US (https://www.arrow.com/en/products/hsfpar003a/alps-electric). Only problem is they're quite specialized, so not many places sell them. The custom PCB was another $10AUD, and the Arduino was about $20. There's a full parts list athttps://github.com/Jcparkyn/dpoint/blob/main/setup-guide.md.

slaucon
1 replies
20h1m

The rolling shutter compensation is pretty cool and isn't something I would have thought of. Did you know that would be an issue from the start or notice it only after you built the rest of the system?

jcparkyn
0 replies
19h46m

I knew it would have an effect (most of the literature for similar projects just uses global shutter cameras for this reason), but wasn't sure how significant it would be. It turned out to be small enough that it usually wasn't super noticeable, but in certain cases it really showed up (e.g., rotating the pen while keeping the tip in one place).

The thing I was most surprised by was how effective my solution was, given that it's a pretty gross approximation of reality. There are lots of much more sophisticated techniques for dealing with it, which I didn't end up needing.

One thing I would've liked to try out is using rolling-shutter-aware PnP [1], which can theoretically estimate pose and velocity simultaneously from one image, by exploiting the rolling shutter distortions.

[1]https://www-sop.inria.fr/members/Philippe.Martinet/publis/20...

joshuahhh
1 replies
13h15m

Impressive work! I wonder: is the inertial tracking sufficient to cover for occasional occlusion of the markers?

jcparkyn
0 replies
8h16m

That depends what you consider "occasional", but for most definitions the answer would be no. I'm sure there are better ways to handle the inertial measurements than what I did, but even so, double integration drift causes errors very very quickly. I doubt it'd be possible to go over a second, and my implementation struggled with anything over ~150-200ms.

However, it does an impressively good job with low camera frame rates. I tested it at 10FPS (discarding every third frame), and the results are barely distinguishable from 30FPS. Below that it starts having difficulty.

denton-scratch
1 replies
1h29m

TFA mentions a gyroscope, but I don't see one in the assembly diagram.

guhidalg
0 replies
1h3m

I believe the PCB has the IMU built into it along with the BT module.https://wiki.seeedstudio.com/XIAO_BLE/

sshkey
0 replies
15h16m

wow, very impressive project! really I love it, I will try to build it one definitly.

sj4nz
0 replies
14h2m

Fantastic. I love that this is brand-independent, like buying a mechanical keyboard.

mncharity
0 replies
17h53m

Nice. Especially the fusion.

Some random-ish thoughts from exploring "a laptop keyboard... with hand pose, 3D stylus, and touch". Adding buttons yields a 3D mouse - but camera coverage can be a pain. Note the body is largely empty (and battery could be slimmed) - I could more-or-less type while holding a slim chopstick/toothbrush-like stylus (even with the markers, and a weirdly big tip). A big tip (sliced from a ping-pong ball sized xmas decoration) could slide fairly smoothly on a ThinkPad keyboard (and gave room for a less compact force sensor, and an extra tip marker). Thin stranded silicone ribbon cable can be string-like flexible - I just tethered the stylus to an arduino to get started.

Hmm... I wonder what the inertial sensor might make of something like a dimpled metal clicker, as a button (or three)?

lagrange77
0 replies
18h59m

1. Very cool project

2. Helpful documentation

3. Nice real world example for the use of a Kalman Filter!

jimmySixDOF
0 replies
18h56m

I'm a big fan of all things 6DOF ! Nice work on the hardware and computer vision pose work but I am almost more impressed by the software surface you are drawing into and able to rotate. Thats interesting and could be used with any tangible user interface control like a finger slider for the same effect. Good project for problem solving skills looks like you nailed it bravo!

Btw the first 6DOF controller I had other (than a hacked WiiMote controller as a ir led Bluetooth receiver [1]) was the logitec mx air which was ahead of its day [2].

[1]https://web.cs.ucdavis.edu/~okreylos/ResDev/Wiimote/MainPage...

[2]https://www.cnet.com/reviews/logitech-mx-air-review/

ezconnect
0 replies
14h26m

Add a laser range finder on the camera and retroreflector on the pen and we have a cheap implementation of a 3D measuring device.