WEBVTT 00:00.000 --> 00:06.000 Good morning. 00:06.000 --> 00:09.000 Welcome to the first am lightning talks. 00:09.000 --> 00:11.000 Great to have you all here. 00:11.000 --> 00:13.000 I'll have a round of applause. 00:13.000 --> 00:15.000 Thank you for coming. 00:15.000 --> 00:18.000 So, my name's Sam. 00:18.000 --> 00:22.000 I'm going to talk about testing no mobile images. 00:22.000 --> 00:24.000 I guess everybody here has a smartphone, right? 00:24.000 --> 00:27.000 Anyone not have a smartphone? 00:28.000 --> 00:30.000 Nobody. 00:30.000 --> 00:35.000 And you're probably aware that you're probably aware that the US tech industry controls 00:35.000 --> 00:39.000 most of the available phone operating systems at the moment. 00:39.000 --> 00:42.000 So, we're quite interested in alternatives, right? 00:42.000 --> 00:45.000 In fact, there's a whole devroom happening this afternoon, 00:45.000 --> 00:48.000 the fossil mobile device is devroom on this exact topic. 00:48.000 --> 00:53.000 So, hopefully, I'll get you interested to join in there. 00:54.000 --> 00:56.000 So, what alternatives are there? 00:56.000 --> 00:59.000 If you want to have a phone which is controlled by you, 00:59.000 --> 01:01.000 and which is all open source, 01:01.000 --> 01:04.000 there's a few different technology stacks. 01:04.000 --> 01:07.000 Probably the most obvious is Android itself. 01:07.000 --> 01:09.000 The core of Android is still open source, 01:09.000 --> 01:12.000 even though a lot of the bits you need actually are close source, 01:12.000 --> 01:14.000 and owned by Google these days. 01:14.000 --> 01:17.000 But the core of its open source and the people working on 01:17.000 --> 01:20.000 replacing the proprietary Google bits with something open source, 01:20.000 --> 01:21.000 called MicroG. 01:21.000 --> 01:24.000 So, one option is Android, and there's things like EOS, 01:24.000 --> 01:28.000 and GrapheneOS, which you can use. 01:28.000 --> 01:31.000 And then, the rest is kind of, you know, 01:31.000 --> 01:33.000 it's a, in the open source world, 01:33.000 --> 01:36.000 we always have different flavors, different tastes. 01:36.000 --> 01:38.000 So, if you like things that begin with G, 01:38.000 --> 01:42.000 there's a G option, which is GTK based, post-market OS. 01:42.000 --> 01:44.000 Let's you run either upstream, no more, 01:44.000 --> 01:46.000 let's you run Fosh, which is, 01:47.000 --> 01:49.000 and there's some, the Purism Phone Shell, 01:49.000 --> 01:52.000 which integrates nicely with Nome. 01:52.000 --> 01:54.000 And then, if you like things that begin with a K, 01:54.000 --> 01:57.000 there's a KDE stack, which is Plasma Mobile, 01:57.000 --> 01:59.000 which is also very good. 01:59.000 --> 02:02.000 And then there's a couple, which also keep based, 02:02.000 --> 02:03.000 but are independent. 02:03.000 --> 02:06.000 So, the KDE plasma mobile thing uses KDE libraries. 02:06.000 --> 02:09.000 Selfish OS is something from scratch. 02:09.000 --> 02:11.000 One of the original open source phone OS is, 02:11.000 --> 02:14.000 actually, even older than Android, I think. 02:14.000 --> 02:17.000 And a bunch who touch or Lumiri, as well as an option, 02:17.000 --> 02:19.000 which is also cute based. 02:19.000 --> 02:23.000 So, is anyone running a G based phone? 02:23.000 --> 02:25.000 Ah, fantastic. 02:25.000 --> 02:28.000 Amazing, that's more than expected, actually. 02:28.000 --> 02:31.000 So, there are many reasons you should try it. 02:31.000 --> 02:34.000 Mainly that you want to have fun, I think. 02:34.000 --> 02:37.000 You, you don't have to write any C++ code, 02:37.000 --> 02:39.000 which for me is a big plus as well. 02:39.000 --> 02:41.000 You can use it to make phone calls. 02:41.000 --> 02:43.000 I have the honor of receiving, 02:43.000 --> 02:45.000 I don't know if it was the first phone call, 02:45.000 --> 02:47.000 but one of the earliest phone calls made from known 02:47.000 --> 02:50.000 in the 2018, that's my colleague Bob, 02:50.000 --> 02:53.000 and he successfully phone me from a Libre M5, 02:53.000 --> 02:55.000 while in Spain. 02:55.000 --> 02:58.000 So, you can use it to make phone calls. 02:58.000 --> 03:02.000 And it's very volunteer driven development, let's say. 03:02.000 --> 03:04.000 There's not a huge amount of investment going into the 03:04.000 --> 03:06.000 known phone stack at the moment. 03:06.000 --> 03:11.000 So, you get to help out and be part of the QA team. 03:12.000 --> 03:17.000 A lot of good stuff happened over the last six or seven years. 03:17.000 --> 03:20.000 No, obviously dates from before this month or even existed. 03:20.000 --> 03:24.000 So, the original designs didn't even consider the idea 03:24.000 --> 03:26.000 of having a tiny vertical screen. 03:26.000 --> 03:28.000 And a few people have done excellent work 03:28.000 --> 03:32.000 adapting GTK and building what began as Libhandy 03:32.000 --> 03:35.000 and is now Libhad waiter, which gives you, 03:35.000 --> 03:37.000 in short, it gives you adaptable widgets. 03:37.000 --> 03:39.000 So, that's a really simple example, right? 03:39.000 --> 03:43.000 If the screen is wide, everything fits on the screen, 03:43.000 --> 03:46.000 and then the same app of the screen is narrow, 03:46.000 --> 03:49.000 bits disappear, and the buttons appear vertically, 03:49.000 --> 03:53.000 so that it fits on a phone screen. 03:53.000 --> 03:58.000 And Libhad waiter is also the core of a lot of no maps. 03:58.000 --> 04:02.000 It's quite new, so non-corrupt developers have rewritten their apps 04:02.000 --> 04:07.000 to use Libhad waiter, and it's the recommended way to build no maps these days. 04:07.000 --> 04:10.000 So, as long as you use Libhad waiter and you make a no map, 04:10.000 --> 04:13.000 you kind of get adaptivity more or less for free. 04:13.000 --> 04:16.000 It's not quite for free because you have to test it, 04:16.000 --> 04:18.000 but you get it more or less for free. 04:18.000 --> 04:21.000 You don't have to maintain two different code paths. 04:21.000 --> 04:24.000 On the shell side, no-m-shel does work in progress 04:24.000 --> 04:26.000 to make it phone adaptable. 04:26.000 --> 04:28.000 Not a lot of it's merged yet, 04:28.000 --> 04:30.000 but there's also a flash that I mentioned before that. 04:30.000 --> 04:33.000 It's a fork, no, it's not a fork of no-m-shel, is it? 04:33.000 --> 04:34.000 I'm lying. 04:34.000 --> 04:38.000 The flash also integrates nicely with no-m and it's built on the same stack. 04:38.000 --> 04:42.000 So there's a lot of good stuff, but we still have the problem 04:42.000 --> 04:45.000 that NGOs are doing a lot of the QA, right? 04:45.000 --> 04:47.000 I mean, do you find non-phone users? 04:47.000 --> 04:50.000 Do you find any bugs? 04:50.000 --> 04:53.000 One or two bugs, maybe. 04:53.000 --> 05:00.000 So, we're still not fully there in terms of testing 05:00.000 --> 05:04.000 in neither in the desktop world nor in the phone world. 05:04.000 --> 05:09.000 On the desktop side, the last 10 or 12 years 05:09.000 --> 05:11.000 there's been some innovation. 05:11.000 --> 05:13.000 The big, the big win has been open QA, 05:13.000 --> 05:16.000 which is a tool that began at open-susa, 05:16.000 --> 05:20.000 and it's now used by several of the major distros. 05:20.000 --> 05:25.000 So open-susa, Debian, and Fedora are all testing with open QA. 05:25.000 --> 05:29.000 And the great thing about open QA is it's an end-to-end test. 05:29.000 --> 05:33.000 So it actually boots the full operating system in a virtual machine 05:33.000 --> 05:35.000 and interacts with it like an end-to-user wood. 05:35.000 --> 05:38.000 So it sends keyboard events, and it reads the screen output 05:38.000 --> 05:42.000 and does image matching to check that the rendering actually works. 05:42.000 --> 05:44.000 So that's very different from a unit test, 05:44.000 --> 05:47.000 where you mock the rendering pipeline and you mock the keyboard 05:47.000 --> 05:49.000 and you mock more or less everything, 05:49.000 --> 05:50.000 and then you test that your mocks work, 05:50.000 --> 05:51.000 and they do, which is great. 05:51.000 --> 05:54.000 But then in the real world, you integrate things together 05:54.000 --> 05:55.000 and it doesn't. 05:55.000 --> 05:57.000 So, end-to-end testing is fantastic, 05:57.000 --> 06:00.000 but it's also kind of the most difficult and awkward to do, 06:00.000 --> 06:04.000 because you are testing everything in a VM, 06:04.000 --> 06:06.000 so it's quite heavy weight. 06:06.000 --> 06:13.000 So in 2021, we set up a simple open QA instance for no, 06:13.000 --> 06:17.000 and I did a talk on that. 06:17.000 --> 06:20.000 Last year or the year before, so I won't go over that again. 06:20.000 --> 06:24.000 But then last year, we ran an internship. 06:24.000 --> 06:27.000 These were two interns that we had Dorothy and Tanju, 06:27.000 --> 06:29.000 and they looked at a few different tests. 06:29.000 --> 06:33.000 One of them they looked at was adding a test for mobile form factor. 06:33.000 --> 06:37.000 And when I spoke this, I thought, how hard can it be, right? 06:37.000 --> 06:41.000 You just make the screen smaller, and it'll just work. 06:41.000 --> 06:46.000 So, firstly, if you set the screen size very small, 06:46.000 --> 06:48.000 or rather, if you set the screen size to a number 06:48.000 --> 06:52.000 that isn't a multiple of 16, then open QA breaks, 06:52.000 --> 06:57.000 because it'll only encode the video if the resolution is a multiple of 16. 06:57.000 --> 07:00.000 So that didn't work. 07:00.000 --> 07:04.000 So we thought, okay, we'll work around that by setting the screen to double the size, 07:04.000 --> 07:07.000 which is actually close to how a phone works anyway. 07:07.000 --> 07:10.000 I'll go into that in a bit. 07:10.000 --> 07:13.000 And we'll set two times scaling. 07:13.000 --> 07:18.000 Guess what? That doesn't work, because mother, the composer, the back-snome shell, 07:18.000 --> 07:22.000 has a hard-coated limitation that it won't do scaling 07:22.000 --> 07:24.000 unless the screen is a certain size. 07:24.000 --> 07:27.000 Which I guess the mobile forks are working around that, 07:27.000 --> 07:29.000 but we're testing the upstream version, 07:29.000 --> 07:31.000 and we were limited to that at the time. 07:31.000 --> 07:33.000 So, couldn't work around that either. 07:33.000 --> 07:37.000 So we thought, well, what if we do a hack inside the OS, 07:37.000 --> 07:41.000 and we pass in some stuff to say, I'm a phone? 07:41.000 --> 07:43.000 That didn't work either. 07:43.000 --> 07:46.000 So we did add a mobile test suite in the end. 07:46.000 --> 07:49.000 I'll show you how that works. 07:49.000 --> 07:51.000 I want to show you a little bit what's remaining to do. 07:51.000 --> 07:53.000 What are the next steps? 07:53.000 --> 07:57.000 My aim for this talk is to make you interested in fixing some of these issues, 07:57.000 --> 08:00.000 so that you don't have to be the test department anymore, 08:00.000 --> 08:03.000 and we can automate it better. 08:03.000 --> 08:06.000 So the first question is pixel density. 08:06.000 --> 08:09.000 This took me a while to wrap my head around, actually. 08:09.000 --> 08:12.000 But it turns out that most desktop monitors, 08:12.000 --> 08:15.000 I think this is true, are around 100 DPI. 08:15.000 --> 08:21.000 So for every inch, unfortunately, it's in inches, 08:21.000 --> 08:25.000 for every inch of screen, you have about 100 pixels taking up space. 08:25.000 --> 08:28.000 For example, if you have a monitor that's, 08:28.000 --> 08:31.000 this is a 24 inch monitor, my example up there. 08:31.000 --> 08:35.000 And so the width is 20 inches. 08:35.000 --> 08:40.000 The x resolution it would be designed for is 1 9 20 pixels. 08:40.000 --> 08:44.000 And if you divide that by 20, you get 96 DPI. 08:45.000 --> 08:48.000 On a phone, you don't get that. 08:48.000 --> 08:52.000 So on a phone, the screen is usually much more dense. 08:52.000 --> 08:56.000 There's an example here that I got from my colleague Adrian, 08:56.000 --> 08:59.000 who actually Adrian did most of this work. 08:59.000 --> 09:03.000 I should credit Adrian with doing most of the investigations I'm going to talk about. 09:03.000 --> 09:05.000 You can be here today. 09:05.000 --> 09:08.000 But thanks to Adrian for doing all of this. 09:08.000 --> 09:11.000 So for example, you look at the Librum 5. 09:11.000 --> 09:17.000 Actually, the UI is scaled by 2. 09:17.000 --> 09:24.000 And the way you do this in the real world is you'd set a display descriptor called the E did. 09:24.000 --> 09:28.000 And you put in the physical width and height of the screen. 09:28.000 --> 09:31.000 And then the rendering pipeline would say, 09:31.000 --> 09:32.000 oh, we need to double. 09:32.000 --> 09:36.000 We need to have twice as many pixels right because the screen is the size. 09:36.000 --> 09:39.000 So QMU at the moment just hard codes 100 DPI. 09:39.000 --> 09:43.000 You can't set the screen width and height on the command line. 09:43.000 --> 09:45.000 It just guesses what it is. 09:45.000 --> 09:50.000 Whatever the x resolution is, it fakes a width based on 100 DPI. 09:50.000 --> 09:52.000 So that would be really nice to fix. 09:52.000 --> 09:53.000 It sounds easy to fix. 09:53.000 --> 09:56.000 Does anyone in here work on QMU? 09:56.000 --> 09:59.000 Ah, fantastic. 09:59.000 --> 10:01.000 Does anyone want to work on QMU? 10:01.000 --> 10:04.000 Very friendly project, right? 10:04.000 --> 10:07.000 Lots of people happy to review patches. 10:07.000 --> 10:11.000 So it's a great first patch if you want to work on this. 10:11.000 --> 10:16.000 Adding widths and height parameters to the display. 10:16.000 --> 10:21.000 And then we'll be able to simulate a phone screen much more accurately. 10:21.000 --> 10:28.000 The next thing, the SM BIOS, which is some data structures that exist, 10:28.000 --> 10:33.000 usually in hardware, and they pass information into the operating system of different types. 10:33.000 --> 10:39.000 The SM BIOS has a field called the chassis type, which lets you say, 10:39.000 --> 10:43.000 oh, this is a server, or this is a handheld device. 10:43.000 --> 10:50.000 And there's some code in the moment that changes behavior based on this value. 10:50.000 --> 10:54.000 I think it doesn't give you power saving warnings if you're on a handheld device. 10:54.000 --> 10:56.000 Something like that. 10:56.000 --> 11:00.000 So that would be really useful if we could set that in QMU as well. 11:00.000 --> 11:01.000 And again, we're almost there, right? 11:01.000 --> 11:05.000 It already has a flag to pass in the SM BIOS Type 3. 11:05.000 --> 11:09.000 We can set the manufacturer, set the version, but we can't set the chassis type yet. 11:09.000 --> 11:18.000 So this is another great opportunity to contribute. 11:18.000 --> 11:19.000 What else is needed? 11:19.000 --> 11:25.000 So to get this to where we can fully test how people really use a phone. 11:25.000 --> 11:29.000 QMU patches are probably the low-hanging fruit I would say. 11:29.000 --> 11:33.000 We also need to think about how to do multi-terge gesture input. 11:33.000 --> 11:38.000 Open QA doesn't have any supports for that at the moment, and it's maybe a little bit more complicated 11:38.000 --> 11:42.000 because we have to think how to record and play back gestures. 11:42.000 --> 11:48.000 I think the QMU bits are there, but making it work nicely in a way that it's nice to all the tests. 11:48.000 --> 11:52.000 It's not there, so that sounds like a fun project. 11:52.000 --> 11:57.000 And then the end goal, which would be fantastic, is actually testing on hardware, 11:57.000 --> 12:00.000 having a hardware rig. 12:00.000 --> 12:05.000 This requires people who want to host and maintain a board farm over time, 12:05.000 --> 12:10.000 which is also really fun. 12:10.000 --> 12:13.000 So that's something, I don't know how we're going to make this happen. 12:13.000 --> 12:15.000 It's going to require investment, right? 12:15.000 --> 12:18.000 Probably from companies that are willing to host these. 12:18.000 --> 12:21.000 It's going to be difficult if they're hosted in people's bedrooms. 12:21.000 --> 12:26.000 But in the long run, I would love it if we have hardware farms distributed around different companies 12:26.000 --> 12:30.000 that work in embedded and work in this space. 12:30.000 --> 12:34.000 And we can actually run tests upstream on real phones and say, 12:34.000 --> 12:38.000 Ah, this commits in whatever library has broken this device. 12:38.000 --> 12:40.000 That would be the dream. 12:40.000 --> 12:43.000 For most of that testing, I'm doing a talk tomorrow afternoon as well 12:43.000 --> 12:45.000 than the distro's devrim. 12:45.000 --> 12:50.000 And I never told you how the tests we worked at the moment. 12:50.000 --> 12:52.000 So we did get something to work. 12:53.000 --> 12:56.000 This is running right now in the Nomoc and QA instance. 12:56.000 --> 12:59.000 And what we do, it's a little bit dirty. 12:59.000 --> 13:04.000 What we do is we boot the machine and desktop form factor. 13:04.000 --> 13:09.000 And then we run some JavaScript code in NomShell to resize the window 13:09.000 --> 13:12.000 so that it's the size that it would be on a phone screen. 13:12.000 --> 13:15.000 And then we take a screenshot of that. 13:15.000 --> 13:20.000 So we're testing the adaptive code paths in LibreWater, at least. 13:20.000 --> 13:25.000 But it's not what I envisioned when we got, when we started working on the test suite, 13:25.000 --> 13:27.000 I hope we'd get a little bit further than this. 13:27.000 --> 13:31.000 And maybe we will this year, still early in the year. 13:31.000 --> 13:37.000 But we are testing stuff at least, so we've made some progress. 13:37.000 --> 13:41.000 My contribution to this, I don't have too much time to dedicate to it. 13:41.000 --> 13:45.000 But my contribution is trying to get people to talk to each other, 13:45.000 --> 13:48.000 which I think is one of the hardest problems in software. 13:48.000 --> 13:52.000 So I run a monthly call that takes place once a month. 13:52.000 --> 13:57.000 Happens, the details are in this wiki page. 13:57.000 --> 14:02.000 Or if you search for, that's quite a long URL isn't it. 14:02.000 --> 14:08.000 If you take a screenshot, then later you'll be able to see the page. 14:08.000 --> 14:10.000 One minute. 14:10.000 --> 14:13.000 So it takes place, all turn it. 14:13.000 --> 14:17.000 But European mornings in alternate European, afternoons, 14:17.000 --> 14:21.000 so that people in Asia can participate in people in the US can participate. 14:21.000 --> 14:27.000 And the idea is to break down the walls between distros and upstreams and different companies, 14:27.000 --> 14:31.000 maintaining test suites and just talk about how can we make really good tests, 14:31.000 --> 14:34.000 how can we improve open QA and so on. 14:34.000 --> 14:36.000 So please join that if you're interested. 14:36.000 --> 14:39.000 Confine me after and I can give you more details. 14:39.000 --> 14:42.000 And also, of course, as the fossil mobile device is different, I'm sure you spotted, 14:42.000 --> 14:43.000 which is this afternoon. 14:43.000 --> 14:45.000 So join that as well. 14:45.000 --> 14:47.000 Thanks to everyone that worked on this. 14:47.000 --> 14:49.000 And that's everything from me. 14:49.000 --> 14:50.000 19 seconds left. 14:50.000 --> 14:52.000 Thank you very much for watching.