This reminds me of the scam reported on here last week, where scammers raised a false large invoice, then invited the mid-level manager to a zoom meeting where the CFO and CEO of the public company were on the call (very public as they needed enough video to deepfake them) and told the guy to approve the invoice.
The gitlab thing is just “harmless” fun but real scams will force chnage - I predict in five years
- every large and maybe small companies will have public private key based approvals (not “I approve” in an email from an iPhone).
- PKIs will then be everywhere (at last!)
- meetings will be recorded and transcribed (which will have huge knock on effects)
- the push back against remote work will be real and large. Not sure what side I am on
I'm at a small company where we've started recording, transcribing and taking minutes of all meetings with AI. Then that gets fed into a RAG system and we can see what's been said when.
If you're doing it in person than all of that information will just float out of everyones heads by the end of the week.
There's a push _for_ remote work because it's actually convenient to find out what you need to do and when it needs to get finished by.
The next thing we're doing is buying everyone an intous screens so we can share whiteboards which is the last thing that in person meetings were good for.
Could you please share details on tooling for this?
Not much tooling I do it all on Linux.
I use audacity to capture the audio stream live, whisper.cpp with person segmentation to extract the text with notations on who was talking and a fine tuned llama model which is trained on data specific to the companies market to extract the minutes. It's a couple of passes for the model to write the prompts to extract the data into minutes.
The RAG system is just untrained sbert weights of every sentence in the recording and minutes shoved into the vector postgres. Queries are @mentions in the company chat.
Basically a bunch of python scripts held together by duct tape and chewing gum.
Very cool. I tried to do this a couple months ago, but really had trouble recording the audio from the call. Are you recording the audio from the call on an external device ie. your mic?
To be clear, my issue was I could only capture the streaming audio of my mic or the people on the call. Never both at the same time.
I have no idea.
I just have a dummy user that joins the meetings so people known when they are being recorded. I imagine I accidentally dodged a bullet there since that user isn't supposed to have audio in and the audio out is directed to audacity and muted otherwise.
When querying, can the system answer with link to the minutes and the recording ? Lack of source citation for facts is a problem with what I've seen of LLMs.
Yes, part of the metadata is the date, person speaking and line number where the hit happens for each sentence if it's a hit in the transcript. If it's a hit of the minutes it's just a link to the date on which the minutes happened.
That's got nothing to do with the LLM used to do the summary but part of the RAG system.
Sounds like a fun project!
Super curious about this bit. Did you use the --diarize or the --tinydiarize flag? How accurate/reliable was the segmentation? Sounds like it was good enough to use at the very least. I had looked into this a few months ago but thought it wasn't good enough yet.
Me too! In particular, assuming whisper segmentation works, how did you recognize and label the individual speakers?
MS teams has this, don't bother cobbling together and maintaining your own solution.
And excel has python!
Jokes aside, excel is the most powerful business software ever. Everyone use it.
Don't bother using it either, have you seen one of those transcripts?
Maybe it works better with an American accent.
This sounds like a recipe for getting people to only speak after careful curation.
Which is a good thing as there is no such thing to unsay.
Its a great way to stiffle innovation, though... Lots of ideas start out as dumb, and then get refined... We have also been using AI, meeting recordings, and summarization to some extent, but I dont see it being used "for all meetings", it will just be very counter productive IMO.
By that logic, company cultures that don't punish dumb ideas will remain innovative though.
I’m pretty sure you could record and transcribe in person meetings just as easily. I’d bet there are several services doing that now but it would also be an incredibly simple project to build in house. The only problem is identifying who is speaking which could be a deal breaker for some uses.
The issue is that in person you constantly have informal "meetings" where information is shared and impossible to record, unless you force everyone to walk around with sound recorders on. Which come to think of it is something that might well be done in some companies I've worked for before.
Trivial, it's a problem that's been solved for a while and I managed to get it working in a couple of days of messing about.
I'm positive that people working remotely can just as easily have informal meetings were they won't be recorded. In fact, I'm absolutely sure that a good segment of the workforce will make sure they have meetings in that informal sense, specifically so the recordings aren't happening.
HHHa, tell me you're middle management without telling me you're middle management.
You can be at the office and forced to do a zoom meeting because some people are in another office/country.
I haven’t had a fully collocated team in 15 years. The only purpose of RTO is tax breaks and stealth layoffs.
God forbid there might be actual people (including developers) who prefer to work in an office with others. Impossible!
They are free to return to the office and work with each other in person then.
Right, so there are more reasons for RTO than tax breaks and stealth layoffs then. Good to know.
RTO usually means mandated, company wide RTO, not "there's an office if you want." So no, not right.
Going by my own experience, everything being remotely or everything being in the office (regularly, but not necessary all the time) works best.
It's hard to make a hybrid approach work, where some people are mostly in the office, and some people are always remote.
However the extra burden falls mostly on the remote people, so I see not much of a reason to disallow people from going remote.
They are not afforded that opportunity. They are required to work with remote people/groups across split sites.
Ok sure, but God also forbid there might be people who don't? Or if someone on there team wants to work in an office with them they should have to go in like it or not?
RTO is clearly not about 'end-employee' preferences, having an office can be about that, but RTO is just come to the office regardless of your preference.
I think the real question in this scenario is "could this have been an email?" Video meetings are disastrously unproductive in my experience. I have no problem with remote workers doing their thing, but being on the production and service side of things in manufacturing, taking time out of my already busy schedule for an hour-long video meeting that could have been summarized in a single-paragraph email makes my day feel like stop-and-go traffic.
It is a remarkably halting experience I used to joke about, but after years of a considerable amount of wasted time that leaves me running to meet deadlines and shipping dates, I have trouble laughing at it anymore.
Unfortunately, the answer to "could this meeting have been an email?" is often “but do you read your email?” I try to always reach for Email first, but if I don’t get a response or see the requested action happen… over to videoconf we have to go :(
Why the pushback against remote work? When it works it works, and it’s cheaper for everyone involved.
I suspect the argument would be you can’t deep fake in-person interactions
Which is just a weird line of thought, leading to "any kind of meaningful decision will only be made when all involved persons are physically in the same room", which results in a chain of meetings between people of different layers and signed papers changing hands.
So in conclusion, the argument is that the existence and threat of deepfakes will cause companies to abandon digital transformation entirely and move back to the 80s...
Paper signatures are of course easy to forge. If anything, I think this will lead to more focus on digital attestation. Maybe even [swallowing a bit of vomit]... blockchain.
I mean, as long as your deepfake is contributing constructively…
I've been deep faking in-person as a person who knows what they're doing for years
I think it depends on the industry. For example, I work in manufacturing and we have a sales team that is in the office once a week, if we are lucky. The disconnect in communication between sales, management (who is also mostly remote) and production is absolutely massive and causing issues that did not exist prior to their current remote status. Examples include that both management and sales are often sluggish to respond when production needs clarification for custom work, and that the typical manager or salesperson's comprehension of the products being sold is atrophied simply because their exposure has diminished in recent years. In the context of our business, it would be beneficial if everyone was in the building working side by side and since only two members or our sales team are "outside" sales, there's not a compelling reason for the rest to be working from home.
On the flip side, my wife does medical coding, which is entirely remote and doesn't require a constant line of communication to management or your coworkers, for that matter, unless there is a problem, most of which can be rectified with an email, chat or Zoom meeting. There is no compelling reason for her to show up to an office to do computer-based work that can be done literally anywhere with an Internet connection.
As the US moves forward with continued remote work, I think it's important that companies be honest with the realities of the market they are in and plan accordingly.
If the company is bigger - then even with forbidden remote work you'll have all different divisions scattered across multiple locations and have the same communication issues as with remote team.
Turns out the issue is not in the remote\office style but in the processes applied to the organization.
There wont be. Not any more than we already have. The comment above you is pure sensationalism based on nothing.
This is not a deepfake, though, to be clear; it's a YouTube video of a real GitLab data science staff meeting.
they were talking about the scam, not the gitlab meeting simulator
Another live video scam I've seen is done via streaming. Scammers will take a long pre-recorded video feed of an on stage interview with a famous person (Elon Musk) and then stream it inside a frame and account that's hyping a crypto launch or something.
Almost all companies have ERP systems with 2FA and compliance teams, there is no more "I approve" emails...
Remote work makes it easier to transcribe every communication. As for PKI you need that anyway and we mostly already have it. We just need a good auth chain for transactions.