WEBVTT 00:00.000 --> 00:12.200 Hello, thank you all for coming, thanks to the thanks to Yaran and the team for inviting 00:12.200 --> 00:15.720 me back here a couple years later. 00:15.720 --> 00:21.400 My name is Eli Malin, you can find me on the at protocol at iME.ly. 00:21.400 --> 00:26.560 This is a continuation of a talk from 2024 called LifeBear Catalyst in the conspiracy to 00:26.560 --> 00:31.520 solve video for everybody for ever, I got a lot of good feedback on that talk, people 00:31.520 --> 00:37.760 seemed to think it was appealing, but I got told, I'm not really sure what you're selling 00:37.760 --> 00:38.760 here. 00:38.760 --> 00:43.160 So, I'm back two years later, I've started a new company, we've got a new product, it 00:43.160 --> 00:48.400 is open source, it's cross-platform, it does a lot of very interesting things, so let's just 00:48.400 --> 00:49.400 go through here. 00:49.400 --> 00:52.480 This is a screenshot that I just happened to take right before we came in here, which is 00:52.480 --> 00:57.480 funny that the growth stream is what popped up on the top of the corner there. 00:57.480 --> 01:03.360 So this is live video for blue skies at protocol, I'm not going to go too deep into the 01:03.360 --> 01:10.280 at-product of stuff today because this is a video talk, but it is an open protocol for decentralized 01:10.280 --> 01:18.080 social, streamplace recently was integrated into blue sky here, so when anybody goes live 01:18.080 --> 01:23.520 to the streamplace you get a little circle around your blue sky avatar there, let's 01:23.520 --> 01:26.040 talk about what we've built. 01:26.040 --> 01:34.760 So we've got very low latency live streaming using H264 and Opus, we've got chats and lots 01:34.760 --> 01:39.440 of other sort of bells and whistles that you would expect from sort of a Twitch style streaming 01:39.440 --> 01:40.440 platform. 01:40.440 --> 01:44.000 The short explanation of this is if blue sky invented the at-product of all Twitter on 01:44.000 --> 01:47.680 the at-product of all reasons, the same thing to make Twitch. 01:47.680 --> 01:53.560 We've got syndication, so you can broadcast your live stream and broadcast anybody can 01:53.560 --> 01:57.680 run a node and re-roadcast your node and syndicate it, as if we're all running our own 01:57.680 --> 01:58.680 little TV stations. 01:58.680 --> 02:03.120 We've got multi-streamings, you can push out to Twitch and YouTube, our Kival, scheduled 02:03.120 --> 02:07.280 streams, web hooks, lots more stuff like that. 02:07.280 --> 02:11.480 And what I'm going to be talking in the most about today is how all content on the streams 02:11.480 --> 02:15.920 is cryptographically signed and has embedded user consent and metadata, so that's what I'm 02:15.920 --> 02:23.080 going to get into most today is talking about the screenplay segmentation format and everything 02:23.080 --> 02:25.440 that goes into that. 02:25.440 --> 02:28.160 So what have we used to make this? 02:28.160 --> 02:29.160 Lots of things. 02:29.160 --> 02:34.240 I want to first shout out the G-streamer folks and anybody here that's worked on Mason. 02:34.240 --> 02:39.160 I think it would be very impossible to do what we're doing without the especially the cross- compilation 02:39.160 --> 02:40.160 features of Mason. 02:40.160 --> 02:45.880 So we have, we have a nominally a go binary but also embeds a bunch of rust libraries for 02:46.320 --> 02:50.720 C2PA signing and I wrote which I'll get into a little bit and then of course a bunch of G-streamer 02:50.720 --> 02:56.040 on top using the go GST bindings. 02:56.040 --> 03:03.320 And yeah, we want all of that to be statically linked Linux, macOS and Windows so we can distribute 03:03.320 --> 03:06.400 it all as a single file binary. 03:06.400 --> 03:10.080 And doing all of that without Mason would have taken me the two years since the last 03:10.080 --> 03:15.360 talk, but thanks to Mason it's been a pretty good deal. 03:15.360 --> 03:17.680 We have two forms of ingest right now. 03:17.680 --> 03:25.600 We have RTNP over missed server and we have the Pion WebRTC library with Whip. 03:25.600 --> 03:28.840 Playback is mostly over Pion WebRTC web. 03:28.840 --> 03:31.840 We also have HLS output that's not very good. 03:31.840 --> 03:36.520 I'm probably going to just like lean on missed server for all the other output formats. 03:36.520 --> 03:44.320 And we have an embedded app so that's live right now on app.stream.plase but that's the same 03:44.320 --> 03:49.000 code based running on Web, iOS and Android and it's bundled in the binary so it's really 03:49.000 --> 03:53.320 supposed to be sort of one file, batteries included, everything you could want to do as 03:53.320 --> 03:56.640 social streaming platform. 03:56.640 --> 04:01.760 So this is sort of I think the answer to some of the questions I was asking in my talk 04:01.760 --> 04:02.760 a couple of years ago. 04:02.760 --> 04:07.360 So it is a decentralized live stream right what does that even mean when we're talking 04:07.360 --> 04:12.360 about decentralized vibe or just sort of decentralized video in general you've got something 04:12.360 --> 04:16.400 like bit torrent right or a lot of these sort of IPFS a lot of these sort of content 04:16.400 --> 04:21.600 addressed systems where you could say okay I'm going to download this video segment or this 04:21.600 --> 04:26.120 video file from somebody and I can be confident that I'm looking at the right thing 04:26.120 --> 04:29.200 because there's a content hashtag to it right. 04:29.200 --> 04:34.160 But live streams haven't happened yet so absent time travel we can't assign a content 04:34.160 --> 04:36.640 ID to a live stream ahead of time. 04:36.640 --> 04:43.640 So what do we do instead is have you generate a key pair, public key and a secret key. 04:43.640 --> 04:48.320 The public key gets posted to your app code account, your blue sky, black sky is all 04:48.320 --> 04:52.320 hosted, I run my own server at IMELite.com. 04:52.320 --> 04:57.800 You can see at the bottom here this is the did key that corresponds to that public key. 04:57.800 --> 05:04.040 And then there's a private key that gets right now put into your OBS stream key and then 05:04.040 --> 05:06.480 you can you can send it into a stream place node. 05:06.480 --> 05:10.960 It's a slightly primitive mechanism I wish there was like a more of a local signing option 05:10.960 --> 05:15.320 but what we have right now with OBS there's sort of like static long lived keys. 05:15.320 --> 05:20.480 So the the simplest way for us to accommodate that list to just put the private key into 05:20.480 --> 05:24.040 the OBS stream key there. 05:24.040 --> 05:29.000 So what happens next you send the live stream in to everybody else I say we slice it up 05:29.000 --> 05:33.680 into one second segments you all know when I say that that I mean we slice it up on keyframe 05:33.680 --> 05:42.200 intervals in each video segment gets into its own MP4 file with G streamer and then 05:42.200 --> 05:48.600 we use the C2PA libraries to give all of these files and embedded signature using the 05:48.600 --> 05:51.800 private key that we generated a second ago. 05:51.800 --> 05:59.080 So this is for those that haven't sort of bumped into the C2PA format this is one piece 05:59.080 --> 06:00.920 of the C2PA spec I really like right. 06:00.920 --> 06:07.560 How do you I've worked with a lot of different sort of decentralized video applications 06:07.560 --> 06:12.040 and a lot of them have sort of oh there's like their video and maybe there's some like 06:12.040 --> 06:15.040 signatures over here and if you download both of those things you can like validate them 06:15.040 --> 06:16.760 against each other. 06:16.760 --> 06:22.360 The cool thing that the C2PA did for this is that it's an embedded signature so it's 06:22.360 --> 06:29.800 basically an inline hash and a signature over all of the other parts of the MP4 file that 06:29.800 --> 06:35.280 aren't part of that hash which is not trivial to do but the cool thing is then the interoperability 06:35.280 --> 06:40.640 is awesome so literally when I say one second and before files you can if you run stream 06:40.640 --> 06:46.520 place on your computer you can see the folder here shout out to the maximum I started 06:46.520 --> 06:50.320 putting them into subject directories shout out to the maximum number of files that you 06:50.320 --> 06:55.000 can have an XT4 file system that I bumped into right away. 06:55.000 --> 07:00.480 So these are all streamplace segments they're all of an embedded signature you could 07:00.480 --> 07:06.000 download them or get to that in a second but you could syndicate these all over the world 07:06.000 --> 07:10.360 download them pass them back into streamplace and you could be confident that you're 07:10.360 --> 07:14.480 looking at the right thing you're looking at the user's actual live stream through the 07:14.480 --> 07:18.520 magic of cryptographic signing right. 07:18.520 --> 07:24.400 We also do embedded segment metadata here so this is a streamplace segment I've pulled 07:24.440 --> 07:33.040 down from me fight in Trabio and Silksong and you can see in here we've got a couple 07:33.040 --> 07:38.600 of C2BA actions here so in this case I'm asserting both I created this live stream and 07:38.600 --> 07:47.960 I published it and we have a standards compliant place.stream.medadata label here so for 07:47.960 --> 07:52.440 example we've got a title associated with this we've got a creator so this is an assertion 07:52.440 --> 07:58.680 that this is associated with my did PLC which is what the app protocol uses for for user 07:58.680 --> 08:06.200 accounts in part and yeah all encapsulated within the np4 file right that was really important 08:06.200 --> 08:11.440 to me that it's not out of band it's so easy to just like upload a file and forget to 08:11.440 --> 08:21.280 include the other stuff so so it's all packaged up together nicely so I've been saying 08:21.360 --> 08:26.720 C2PA so it's just in terms of things I know video people will be interested in how 08:26.720 --> 08:35.840 we can untie I'm actually okay great so this is the allowed signature algorithms in the 08:35.840 --> 08:42.480 C2PA specification it's got a lot of the greats a lot of the big ones in there but what 08:42.480 --> 08:46.840 it doesn't have is the most common signing method that the app protocol uses which is 08:47.080 --> 09:00.760 C2PA 6K1 signing which is to say they have this one here which is 256 bit ECDSA signing over 09:00.760 --> 09:10.200 the SEC the the P256 curve but not the K256 curve because that's this is the this is the evil 09:10.280 --> 09:16.360 math that's used in blockchains and stuff so we wouldn't want to have that one right so I don't 09:16.360 --> 09:22.440 love that so I'm sort of talking up some of the C2PA formats that I really like but I don't like 09:22.440 --> 09:26.600 that piece of C2PA so we sort of like we're not spec compliant with this this is the other thing 09:26.600 --> 09:32.760 I don't like so Adobe's in the process of lobbying governments to adopt legislation that says 09:32.760 --> 09:43.800 like all content all especially AI generated content has to be C2PA signed and not a terrible idea 09:43.800 --> 09:50.200 in practice obviously like deepfakes are a big problem and we need to you know the idea of if 09:50.200 --> 09:54.680 you're proactively generating AI content that you have to label it as such it's not like a 09:54.680 --> 10:00.360 really advocating for it but that's not like a ridiculous regulation but this is Adobe's motivation 10:00.360 --> 10:04.600 here for this right is that they can sell a lot of these all of a sudden there's a bunch of 10:04.600 --> 10:09.560 European governments that have this new regulation passed and oh by the way here's like the one 10:09.560 --> 10:13.960 piece of software that's compliant and you can actually do all of this so don't love that piece oh 10:13.960 --> 10:17.640 also they want us to pay for sorts again which is really funny to me like years and years after 10:17.640 --> 10:22.600 let's encrypt that we're gonna all start paying $107 a month for document signing certificates to 10:22.600 --> 10:30.280 did you search which I don't think that's probably gonna happen so yeah what I can announce here 10:30.280 --> 10:36.920 today we are in the process of standardizing a fork of the C2PA specification and collaboration 10:36.920 --> 10:46.120 with the IPFS foundation specifically there the the dazzle project that has standardized a bunch of 10:46.120 --> 10:54.280 C4 profiles that are used in the at protocol a bunch of other things so yeah that is we're just 10:54.280 --> 10:57.800 in the process of kicking out that project it's been in the last couple weeks but if you're 10:57.800 --> 11:04.360 interested in building sort of totally free alternative to some of these other structures I would 11:04.360 --> 11:10.680 encourage you to check out to dazzle it dazzle that in there okay I need to help on a couple things 11:10.840 --> 11:18.840 part of the reason I came here I'm on time yeah the first thing is I appeal to you all as engineers 11:19.560 --> 11:25.080 imagine that you've made a video product and every time people use it the video looks really bad 11:25.640 --> 11:30.360 and you have to go and chat and everybody's say oh well we use web RTC playback so you can't have 11:30.360 --> 11:36.040 beef frames so you need to enable advanced settings and type beef frames equals zero into x to 11:36.040 --> 11:41.160 six four options and then you have to do this over and over and over again for dozens of different 11:41.160 --> 11:47.480 users biggest single problem that we have right now there's a solution to this which would be 11:47.480 --> 11:52.920 get streamplace included in the OBS default list of drop-down that you can pick OBS from there and 11:52.920 --> 11:56.760 then we can say have some hard-coded video settings in there and make it a lot easier for people 11:57.320 --> 12:02.360 we've been turned down twice for not being popular enough but the policy says they can make 12:02.440 --> 12:07.640 exceptions so I appeal to you all if anybody has any poll with the OBS team that could help us get it 12:07.640 --> 12:15.080 on there oh this is such a painful one for us the other project that we'll be taking on as part of the 12:16.200 --> 12:22.840 S2PA simple standard for provenance and authenticity is deterministic muxing so we've got all these 12:22.840 --> 12:29.720 one second files they've got embedded signatures but having many many thousands of them on your computer 12:29.800 --> 12:37.080 kind of stinks so what we'd like to do is combine a bunch of them into like say one hour and before 12:37.080 --> 12:41.800 file which would be a much better archival format but in order to do that and still have the 12:41.800 --> 12:47.800 cryptographic signatures the intact what we need is I said deterministic we don't do need deterministic 12:47.800 --> 12:54.920 nuxing but we also need reversible muxing right that we can take in this one hour like this one hour 12:54.920 --> 12:59.640 file split it back into one second files validate all of the signatures in those one second files 13:00.360 --> 13:05.400 so you can get out exactly what you put in it's spent two weeks on it haven't quite cracked it yet 13:05.400 --> 13:09.400 I think this is probably going to take rust and wasm but we're going to probably have some like 13:09.400 --> 13:16.120 funding for people to work on this kind of thing through the IPFS foundation and some other folks 13:16.120 --> 13:24.360 I'm going to try and get working on it so yeah if anybody's interested yeah if anybody's 13:24.360 --> 13:27.800 interested you want to read more about it I've got some blog posts written on how this works at 13:27.800 --> 13:32.360 box.stream.plase talking about becoming the video primitive for the next thousand years of social 13:33.320 --> 13:35.320 video thanks very much 13:42.040 --> 13:44.040 we have time for questions 13:52.840 --> 13:57.480 I have tried medium medium tx oh yes so the question was why missed server for our 13:57.480 --> 14:04.600 kmp ingest rather than uh nx rtmp or media mpx um a big requirement would be the static linking 14:04.600 --> 14:10.040 so linking all of nginx into this probably possible but not something I was super excited about 14:10.040 --> 14:15.880 I tried media mpx we might have gone with them they only recently this is just like a go technical 14:15.880 --> 14:21.160 thing but media mpx until recently had all of their rtmp stuff in an internal sub directory so it 14:21.160 --> 14:26.120 couldn't be included as a library that has changed relatively recently like in the last year 14:26.680 --> 14:31.560 and I have played around with using the embedded media mpx server and just like hit a couple 14:31.560 --> 14:36.040 problems with it so and we already have like a really nice working one with miss server so that's 14:36.040 --> 14:40.040 that's a short answer but we do use other media mpx stuff internally and I'm interested in that project 14:42.600 --> 14:49.480 hey how far back just the authentication code from this like how close to the cameras you get 14:49.480 --> 14:55.480 from when you start signing it great question right right now this sort of the signing starts from 14:55.800 --> 15:03.400 yes so the question is sort of uh this is a good C2PA question right which is how far 15:03.960 --> 15:07.880 how far back does the attestation chain go right do we go all the way back to the original camera 15:08.840 --> 15:13.240 right now OBS doesn't have any integration like that so most of our users that are using that 15:13.240 --> 15:18.120 that's sort of where the the source of truth begins and where the signing begins 15:19.160 --> 15:23.800 I you know as somebody working on all of the stuff I'm very very interested in the idea of 15:24.440 --> 15:34.120 having compatibility with C2PA cameras or having like or taking multiple streamplace streams 15:34.120 --> 15:38.840 and combining them in a way that keeps all of the the manifest and provenance intact so 15:38.840 --> 15:42.280 I would love to have all of that stuff that's just a matter of like instrumentation on the software 15:42.280 --> 15:43.800 that people usually use for these things 15:44.440 --> 15:48.440 hey 16:01.720 --> 16:07.800 yeah it's it's just a matter of when you get into it's it's nice for this syndication step I have 16:07.800 --> 16:13.400 back here it's nice to have this sort of like well defined chunk uh sorry didn't repeat the 16:13.400 --> 16:17.880 question the question was why one second chunks as opposed to like a continuous stream that you're 16:17.880 --> 16:24.040 that you're rehashing you know one uh so like one advantage here as you get sort of very bit 16:24.040 --> 16:28.840 torrent like properties where if I want to watch a stream I could be downloading it concurrently 16:28.840 --> 16:33.160 from four different sources and for that reason it's nice to have sort of the self-contained 16:34.120 --> 16:38.520 and well-defined chunk um I think we will probably especially as we get into like the 16:38.520 --> 16:44.120 concatenation stuff that I was talking later combining videos I'm looking maybe at changing the 16:44.120 --> 16:48.760 segmentation format because you can't just squish in before files together and then use Iro 16:48.760 --> 16:53.240 and Blake 3 and then like you're done and you've got this like really nice sort of a format for 16:53.240 --> 16:59.080 everything um so uh yeah more more to come on that but yeah it's like the bit torrent properties 16:59.080 --> 17:04.760 like I was describing I guess what do you see the thing the issues of the next thousand years 17:07.320 --> 17:12.200 that you know but I don't know it's gonna happen in the next thousand years but uh the question 17:12.200 --> 17:17.320 is uh like what are the issues that are gonna come up in the next thousand years what I'm going 17:17.320 --> 17:22.280 for here is like a thousand years from now how are you gonna know that somebody's twitch stream 17:22.280 --> 17:26.840 or whatever right like it's just like there was like a video on a website and we like to 17:26.840 --> 17:32.680 picture of it or something um as opposed to uh this is I think the the nice thing about content 17:32.680 --> 17:37.080 addressing and signed primitives and things like that is you could legitimately look at this 17:37.080 --> 17:41.320 I don't know a thousand years but hundreds of years from now and still be oh yeah that signature 17:41.320 --> 17:47.000 is intact even if the private key hasn't survived and and we know we're looking at authentic content 17:47.160 --> 17:57.480 I have uh yeah I think we're I think we're at time uh we'll do one more real quick 17:59.480 --> 18:04.360 so if you were anyone who's there can't be where I would stream you can push to that 18:04.360 --> 18:10.840 important right that's correct that's correct the the the the question was uh can anybody 18:10.840 --> 18:16.280 with the R&P's key is uh R&P the this like private key that I talked about earlier can anybody push 18:16.280 --> 18:21.320 to that uh the answer is actually yes right now but we have a open pull request working on that 18:21.320 --> 18:31.160 to make it so um to separate out these two actions uh so that the created action and the published 18:31.160 --> 18:35.880 action are actually signed by two different things so the creator if you get captured that one 18:35.880 --> 18:40.680 key you get the creator key but that would be only valid for a certain publisher that would be 18:40.680 --> 18:44.920 controlled on the server or whoever else is actually publishing the live stream and only with those 18:45.000 --> 18:51.160 two things combined uh would you be able to actually stream somewhere so you know still the static 18:51.160 --> 18:55.240 credential it's still like not ideal but that's another layer of security if you really really 18:55.240 --> 19:00.200 care you should run stream place locally and use like BKS11 and do local signing of your segments 19:00.200 --> 19:04.040 before you send them out that's like the really good way but this is what we need to do for OBS compatibility 19:04.040 --> 19:11.080 right now cool thanks everybody 19:14.920 --> 19:19.920 .