WEBVTT 00:00.000 --> 00:11.440 Hi everyone, I'm Ryan. I'm going to give a talk on basically formalizing dependency resolution 00:11.440 --> 00:16.400 and trying to show that every ecosystem is kind of doing the same thing underneath the 00:16.400 --> 00:22.480 hood. As the title says, is a formal model so there will be a bit of Greek but I'm going 00:22.480 --> 00:27.920 to try and get the intuition across as well. So there are a lot of package managers out there 00:27.920 --> 00:33.680 as I'm sure everyone in this room knows. Every language and operating system has something else 00:33.680 --> 00:41.040 going on. They each have subtly different semantics for dependency resolution. This fragmentation 00:41.040 --> 00:46.400 prevents projects written in multiple languages from managing their dependencies in a unified 00:46.400 --> 00:54.240 way and the mechanisms for depending on system dependencies are often sort of ad hoc and 00:54.240 --> 01:02.960 unversioned. And that means there might be, you don't have filled knowledge of your dependencies 01:02.960 --> 01:06.320 basically like your language package manager doesn't know what's going on with you system 01:06.320 --> 01:14.320 package manager. So I present to you the package calculus, which is a small formalism trying to capture 01:14.320 --> 01:22.480 the course semantics of dependency resolution. But there's hopefully able to capture the diversity 01:22.480 --> 01:29.600 out there in the world which I'll get onto. So we have n as the possible package names v as the 01:29.600 --> 01:35.120 possible package versions p as the Cartesian product of these two sets which means a package as 01:35.120 --> 01:41.440 a name version pair and r as all the packages which exist for example in a repository. 01:42.800 --> 01:51.040 We define the delta as a dependency relationship from a package to a package name and a set of 01:51.040 --> 02:01.040 versions that you depend upon that package with. As in I depend on library A with versions 1.1 to 02:01.040 --> 02:15.520 two for example. And given a set of the dependency relation and a root package R we define a resolution 02:15.600 --> 02:26.240 s as valid if it one includes the root to satisfies the dependency closure. So for every package 02:27.120 --> 02:35.200 in the resolution for each dependency of that package there has to exist a package in the resolution 02:35.200 --> 02:44.400 that has the same name and a version that is in the version set. And also there is this criteria 02:44.560 --> 02:49.440 version uniqueness. So if there's two packages in the resolution with the same name they have to have 02:49.440 --> 03:03.440 the same version. Is that making sense so far? Okay. So for example we we can have this dependency 03:03.440 --> 03:09.360 relation where we have a 1 depending on b1, a 1 depending on c1, b1 depending on d with versions 03:09.680 --> 03:19.920 2 and 3. And the solution to this is to select d2 and all the other packages. And we can 03:20.720 --> 03:27.680 diagram this as a hypergraph essentially where we have an edge from a package to a set of packages 03:27.680 --> 03:38.400 it depends upon. This is NP complete finding the resolution to satisfy a root dependency relation. 03:39.280 --> 03:47.760 The proof is via a reduction from three set which I will admit to save on a bit of time. 03:49.040 --> 03:55.040 So as you probably know modern package managers often employ sat to solve this problem. 03:57.200 --> 04:05.680 So an example of how you could do that in the package calculus is using a variable to represent 04:05.760 --> 04:10.640 each package in the resolution which is true if the package is included in the resolution. 04:10.640 --> 04:15.440 So a variable for each package in the repository which is true if it's in the resolution. 04:17.120 --> 04:25.440 And then the conjunction of every dependency where we say we either don't have the package 04:26.400 --> 04:36.880 which is depending upon the other package or we have any package which has the name we depend on 04:36.880 --> 04:44.400 with a compatible version. And we can we can translate this to conductive normal form for a 04:44.400 --> 04:49.840 set solver. And a similar thing for a version uniqueness if we have two packages in the 04:50.400 --> 04:56.880 repository they they they they can't be in the resolution with the same version. 04:58.240 --> 05:03.680 This has pretty bad complexity for a normal sat solver but a sat solver with an at most one primitive. 05:04.240 --> 05:07.760 It's fine and this works in the real world. This is nothing super novel. 05:10.800 --> 05:17.120 So we've described compatible versions as a set so far but in the real world you 05:17.680 --> 05:25.520 can normally express version dependency versions with a range and this is predicated on having an 05:25.520 --> 05:32.560 ordering for these versions. So this is basically to say that given a range of versions we can 05:32.560 --> 05:42.160 collapse this to a set given a repository pretty intuitive. And we can talk about the freshness 05:42.240 --> 05:48.240 of a resolution given this version ordering. So most ecosystems prefer to use the highest 05:48.800 --> 05:55.680 compatible version. And we can make the sat solver prefer this by putting the freshest versions 05:55.680 --> 06:00.400 earlier in the clause. And if it tries the clause variables in order it will prefer the more 06:00.400 --> 06:04.880 fresh versions. So I think those systems don't do this. I think go prefer to the minimum version. 06:05.440 --> 06:12.000 I'll get onto go in a little bit. There is no guarantee that there is actually a single maximum 06:12.000 --> 06:18.160 resolution. For example in this scenario we have a depending on b with versions 1 or 2 and c with 06:18.160 --> 06:25.360 versions 1 or 2. But b2 depends on c1. So we can't select c2. So there's a trade off there. 06:25.360 --> 06:33.840 You can either select b1 and c1 or b1 and c2. There is some work exploring this in a little 06:33.840 --> 06:38.160 more detail but we're just focusing on the satisfaction of dependency resolution. 06:39.040 --> 06:46.880 So can we avoid the empty hardness of this problem of finding the resolution to satisfy 06:46.880 --> 06:54.880 dependencies? Well go has an approach called minimum version selection where we only specify 06:54.880 --> 07:00.240 the minimum version by infer dependency. So we only describe one version which is the minimum 07:00.240 --> 07:06.880 we can depend upon. And the maximum is implicit up to the next major version. So 07:08.880 --> 07:16.560 it's a linear time traversal of the dependencies in order to find a resolution. You just take 07:16.560 --> 07:24.640 the maximum of all the minimum version byns. This is reliant on strict semantic versioning. 07:27.360 --> 07:33.840 There is no facility to express a maximum other than the next major version bump. So we have to 07:33.840 --> 07:39.760 always have compatible minor and patch versions. Only major versions can break the APIs. 07:39.760 --> 07:44.640 But this isn't suitable to all ecosystems. For example, a camo which is a language 07:44.640 --> 07:50.400 and more familiar with. Any API changes potentially breaking due to module 07:50.400 --> 07:54.240 includes of types. So getting into the technical details. Basically any API change could 07:54.240 --> 07:58.800 be breaking. So there's no meaningful distinction between major and minor versions. And we 07:58.880 --> 08:07.840 devolve into expressing precise dependencies. As in nix's model, we can only depend on a singular 08:07.840 --> 08:15.680 version. So we can't express dependencies on multiple major versions. Make sense? Okay. 08:16.960 --> 08:22.800 So now that I've described the core package calculus, I'm going to try and convince you that this 08:22.800 --> 08:30.400 is expressive enough to model most of the all. I'll say all of the, improve me wrong. I'd 08:30.400 --> 08:36.560 love to be improve me wrong. All the package manager dependency resolution semantics out there. 08:37.520 --> 08:45.840 So the first one is very simple. Version formula. We have Pi which is a version formula such as 08:45.840 --> 08:51.520 greater than 1.2.3 less than 2.0. As I mentioned before, we can basically collapse this to a set 08:51.520 --> 08:56.880 given a repository. We just take all the versions in that repository which is in that formula. 08:58.160 --> 09:04.880 The next one is conflicts. So many package managers support anti dependencies. Let's say I have a 09:04.880 --> 09:09.040 fork of a project which is packaged under a new name. There might be some reasons why that can't 09:09.040 --> 09:16.240 be co-installible with another package. So apps, RPM and Opam also support a conflict from one package 09:16.320 --> 09:26.720 to another package name. So for example, Pi gamma and Vs means P cannot be constable with N where 09:26.720 --> 09:34.000 any V is in Vs. You might think given we have nothing in the core calculus to support this. 09:34.000 --> 09:40.960 This might not be possible to represent, but we can use a trick to encode this in the core calculus. 09:41.760 --> 09:47.920 Oh yeah, basically this is the semantics of the resolution in the conflict calculus where 09:48.800 --> 09:55.840 if we have a conflict on an NVS then we can't have a V existing in the resolution with a V in Vs. 09:58.080 --> 10:04.640 But we can reduce this to the core calculus using these virtual packages. So we create a virtual 10:04.640 --> 10:14.320 package called NVS. So this is for the package N with the set of versions Vs with two versions, 10:14.320 --> 10:22.000 zero or one and the package that is in conflict with NVS depends on this package with version one 10:22.560 --> 10:28.800 and every NVS depends on this package with version zero. And this means we're using the 10:28.800 --> 10:36.960 version uniqueness in order to give conflicts. So if A1 depends upon, sorry, conflicts with B1 of 10:36.960 --> 10:42.560 E2 we can encode it as this in the core calculus. And what this means is package management 10:42.560 --> 10:54.720 ecosystems that don't support conflicts can solve ecosystems that do. The next one is concurrent versions. 10:54.720 --> 10:58.080 So as mentioned in the previous talk, there's this diamond dependency problem in 10:58.080 --> 11:03.600 partition management where you depend on two packages each of which depend upon another with 11:03.600 --> 11:10.160 different versions. No valid resolution for this exists under our version unit is criteria. 11:13.680 --> 11:19.360 But some package managers allow multiple versions of the same name to exist in a resolution. 11:20.000 --> 11:26.000 Cargo allows packages to coexist only when they have different major versions. 11:27.200 --> 11:34.640 NPM allows any different version to coexist. NIX using hash-based paths allows 11:36.160 --> 11:49.200 anything to coexist as well. So we can define a granularity function which maps 11:49.200 --> 11:56.320 versions to a granularity. So for Cargo we take the major version for NPM and NIX we can take 11:56.880 --> 12:02.800 the granularity of the version as the version itself. And for the core calculus we can we can 12:02.800 --> 12:08.880 define your granularity of every version as being the same which is epsilon here. And we define 12:08.880 --> 12:15.520 a concurrent version resolution as instead of version uniqueness. It's the same as the core 12:15.600 --> 12:20.720 calculus but instead of version uniqueness we have version granularity. So for any two packages 12:20.720 --> 12:27.440 that exist in the resolution the versions not being the same implies that their granularity 12:27.440 --> 12:34.800 is also not the same. So if we have two versions of the same major version in Cargo then this would 12:34.800 --> 12:38.960 not apply. But if they are different major versions then this would apply and it would be a valid 12:38.960 --> 12:49.760 resolution. Okay sorry for all the Greek. We can readjust this to the core calculus using 12:51.920 --> 12:58.320 again made up packages. There's a lot of Greek here but let me just give you the intuition. So we have 13:00.000 --> 13:05.920 an example where we have the dumb-independency problem. We have A1 depending on B1 and C1. B1 depends 13:05.920 --> 13:17.200 upon D1, D2 and D2.0.1 and similarly C depends on the two's and three. And under the core calculus 13:17.200 --> 13:23.040 there wouldn't be a valid resolution for this but we can encode this from the extension to the core 13:23.040 --> 13:33.040 using these proxy packages. So the key thing is here we have a proxy package for the C the C 13:34.000 --> 13:44.880 package at version 1.0.0 to package name D and then we have the versions as the granularity function 13:45.600 --> 13:58.240 of its dependencies and then D2 can depend upon D2.0.0 and 2.0.1 as well as the three major 13:58.240 --> 14:04.320 versions. So we use an intermediary in order to be able to depend upon packages with different 14:04.320 --> 14:17.120 major versions. Okay and the point of this, the point of this is any ecosystem which doesn't support 14:17.120 --> 14:28.160 concurrent versions can still interoperate with another ecosystem that does. Okay. Another weird 14:28.720 --> 14:35.040 thing out there is features. These are basically parameterized dependencies. So cargo and 14:35.040 --> 14:40.880 pip support optional functionality of a package being enabled by depending upon it with a 14:40.880 --> 14:48.320 feature in cargo or the called extras in pip. So the one of the key points here is that 14:48.320 --> 14:52.800 the resolver unifies features across the resolution. So if I have package A depending on the package 14:52.800 --> 15:00.320 B with feature alpha and package C depending on package D with feature beta, alpha and beta will be 15:00.320 --> 15:10.960 unified and B will be built with features alpha and beta. Okay. We define a parameterized dependency 15:10.960 --> 15:18.720 where you can depend upon a package with a set of features and we define additional dependencies 15:18.800 --> 15:23.760 that can be enabled by the enablement of a feature. So if we have package P enabled with feature 15:23.760 --> 15:34.000 F we can add an additional dependency. Okay. Okay. Can I get a show of hands are people following 15:34.000 --> 15:43.280 along? Okay good. Now there is a lot of Greek for this one but I'm again going to just give 15:43.360 --> 15:58.160 you the intuition. Okay. The resolution criteria are for features that you depend upon a package 15:58.160 --> 16:02.800 with that package to be provided with those features. Unsurprisingly and if a package is provided 16:02.800 --> 16:09.360 with a feature then you depend upon the additional packages enabled by that feature. So if we have 16:09.360 --> 16:17.440 a package A1 which depends on B1 and C1, B1 depends on D1 with features alpha and beta. 16:17.440 --> 16:24.960 C1 depends on D1 with features beta and D1 if it is dependent upon with alpha depends upon 16:24.960 --> 16:31.760 the additional package E. So to give you this could be a functionality like support for PNG images 16:31.760 --> 16:38.320 for example and if you depend upon this package with the PNG functionality it needs to link in 16:38.320 --> 16:44.400 a library to support PNG images. And then maybe this is JPEG images so you need a JPEG library. 16:45.840 --> 16:52.080 So D1 depends on E1 if it is dependent upon with alpha and F1 if it is dependent upon with beta. 16:52.800 --> 16:57.200 And we can register this to the core calculus again with these intermediate packages. 16:57.760 --> 17:08.080 We create a virtual package for every feature that D1 supports. So we have D1 with feature alpha 17:08.560 --> 17:16.080 and D1 with feature beta. And when you depend upon a package with a feature you include that 17:16.080 --> 17:21.920 as depending upon this virtual package and then that itself depends upon D1 and any additional 17:21.920 --> 17:28.000 packages enabled by that feature. Now if you go into the weeds of the maps you run across some 17:28.000 --> 17:37.440 complexities with different versions of packages with different features just if you want you can go and 17:37.520 --> 17:41.120 look at the slides and dig through the maps and convince yourself that this is right but 17:41.840 --> 17:46.160 basically this encoding supports different versions of packages and different features. 17:47.360 --> 17:53.040 Okay. Now we also have something called peer dependencies in the MPM world. 17:53.680 --> 18:00.000 This is where a child tells its parent what versions of a dependency it can depend upon. 18:00.000 --> 18:05.520 Okay. This is used for plugins so like if I have a if I have a package and I 18:06.080 --> 18:11.600 and I depend upon some JavaScript framework and a plugin for that framework. 18:11.600 --> 18:18.320 That plugin can constrain the versions of the system. Yes. Yes. 18:21.440 --> 18:27.120 So we model the legacy peer dependencies behavior where a peer dependency is only included if 18:27.120 --> 18:30.560 it's parent also depends upon it so when you depend on plugins they don't implicitly 18:31.120 --> 18:37.680 bring in the the framework which it uses but you can also model the new behavior where a plugin 18:37.680 --> 18:40.880 brings in its framework when you depend upon it with some minor tweaks. 18:42.880 --> 18:48.640 So we define theta as this peer dependency model where if we have a package which 18:49.440 --> 18:58.240 peer depends on a M with versions Vs this means package envy has a peer dependency on M with versions Vs 18:59.200 --> 19:06.400 so its parent will need to satisfy the Vs version constraints there. 19:07.280 --> 19:11.600 So we have this parent function pi which maps the package to its parent. 19:11.600 --> 19:17.200 I've admitted for the sake of brevity how that is created but you can you can do this in the 19:17.280 --> 19:26.640 gore calculus. The the the the the criteria is if a package peer depends upon another package 19:27.840 --> 19:35.840 second time poster and the parent of that package also depends upon that peer dependency 19:35.840 --> 19:44.400 then the package used to satisfy this dependency must satisfy the disjunction of the peer dependency 19:44.480 --> 19:47.360 version constraints and the parent dependency version constraints. 19:52.480 --> 19:56.480 Here is an example of how we can register to the core calculus I've admitted the maths here 19:57.040 --> 20:05.280 just giving you the the an example we have A1 which depends upon this framework C and we have a plugin 20:05.280 --> 20:12.960 B1 which depends upon the the the the the framework C with these particular versions and you notice 20:13.040 --> 20:19.040 the only overlap of these two version constraints is C2 we can record this in the core calculus with 20:19.040 --> 20:26.880 these two proxy packages we have this representing the version selection of C2 for this peer dependency 20:26.880 --> 20:35.840 A1 on C and we have A1 pending upon that so other packages can depend upon B as well 20:37.200 --> 20:42.800 which might have different peer dependencies so this is why we need this proxy package and basically 20:42.960 --> 20:49.120 this package here gives us the unification of these two version sets and we can support peer 20:49.120 --> 21:00.560 dependencies in the core calculus. I've got five minutes left right something like that okay I'm going to 21:01.920 --> 21:11.600 I'm going to I'm going to skip past some stuff so we can we can support we can support package formula 21:11.600 --> 21:21.680 so opam and debion allow you to say I depend upon package A or package B or package A 21:21.680 --> 21:27.920 and package B the conjunction is simpler you just depend upon both of them but the 21:27.920 --> 21:33.280 dist junction requires you to introduce a proxy package so we have this hypothetical dependency here 21:33.280 --> 21:39.360 where A1 depends upon a dist junction of two conjunctions and we introduce a proxy package one 21:39.440 --> 21:44.960 is the left one is the right and the version selection basically gives you the dist junction we 21:44.960 --> 21:54.240 either select or the left or the right and the the left depends upon B2 and C1 the right depends 21:54.240 --> 22:00.640 upon B1 and not C1 notice this encoding is the same as the conflict encoding from before 22:00.960 --> 22:10.240 now these formulas can also variables in them sometimes such as the operating system or 22:11.440 --> 22:17.600 like a with test or a with doc we can introduce proxy that we can introduce virtual packages to 22:17.600 --> 22:23.840 represent these variables in the dependency resolution this is different from how it's implemented 22:23.840 --> 22:27.760 in a lot of package manager ecosystem so it's basically a pre-processing step where you filter 22:27.760 --> 22:32.560 the packages that don't match the operating system you're on for example but by putting these 22:32.560 --> 22:36.640 variables in the dependency resolution it can allow us to do things like solve for the particular 22:36.640 --> 22:42.720 operating system we might want to use so I can say I have a I can spin up a debion vm or a 22:44.800 --> 22:48.480 I don't know another vm and I can solve for which operating system might provide the system 22:48.560 --> 22:59.680 dependencies I need there there's also this idea of virtual packages in debion so 23:00.560 --> 23:08.160 two packages can provide something like an SSH server and there is another mechanism for 23:08.160 --> 23:13.040 including this in the package calculus where you create a virtual patch again there's a patch in here 23:13.680 --> 23:24.240 so sometimes there's dependency cycles in packages managers different ecosystems have different ways 23:24.240 --> 23:31.040 of breaking these dependencies um opam has a post variable that says this particular 23:31.040 --> 23:39.680 cyclical can be broken here at I think just arbitrarily breaks at a certain point but the most of the 23:40.640 --> 23:48.080 cycles are in the core of the the core dependencies in debion and I think they are provided 23:49.280 --> 23:54.640 in a bundle so it's not that much of an issue mpm you have cycles everywhere the language breaks 23:54.640 --> 24:01.360 that the package manager doesn't care it just fits the files in the file system there is also 24:01.360 --> 24:06.160 this concept of optional dependencies in opam which took me a while to figure out how we can 24:06.160 --> 24:11.040 represent in the core calculus but it turns like you don't have to um optional dependencies basically 24:11.040 --> 24:17.040 say I use a package if it's present in the resolution but otherwise I don't care it's opam's way 24:17.040 --> 24:23.040 of doing features essentially um it's a little less elegant but we can basically just represent it 24:23.040 --> 24:25.840 in the build order it doesn't affect the resolution semantics as tall 24:26.800 --> 24:40.560 um next has basically no um constraints on versions you depend on a specific package right 24:40.560 --> 24:46.320 and you you get the possibility to depend on different packages by parameterizing derivations in the 24:46.400 --> 24:54.880 next DSL um so the the the actual resolution is either completely turning complete or non-existent 24:54.880 --> 25:01.200 depending on your perspective um so we can represent the the sort of core derivation building part 25:01.200 --> 25:05.360 very simply by just constraining the versions set to be the size of one but this means that next 25:05.360 --> 25:12.640 isn't expressive enough to do the the the the dependency resolution the other ecosystems do 25:12.960 --> 25:19.360 um and this is why you have like all these extra next tools out there um that do versions 25:19.360 --> 25:23.840 solving in the ecosystem specific tooling and then programmatically generate the next renovations 25:23.840 --> 25:32.400 with the bunch of versions pinned um so I'm kind of interested can we can we can we solve dependencies 25:32.400 --> 25:40.160 across ecosystems in a more unified and holistic way um and I I'm thinking my I'm proposing that the 25:40.240 --> 25:45.040 core calculus is basically the minimum you need in order to support all these weird and 25:45.040 --> 25:50.640 wonderful features like that are in the wild and we can create a language as the lingo franca 25:50.640 --> 25:56.000 for these ecosystems so if you want to talk to open from cargo you can go through pack 25:56.000 --> 26:01.760 reduce it down to pack expand it out for the features functionality support and then we can do 26:01.760 --> 26:19.760 cross ecosystem resolution um yeah any questions yeah the implementation is a working 26:19.760 --> 26:25.520 example or something I think I've got a bunch of hacks like they're translating various 26:25.520 --> 26:33.280 ecosystem repositories to open repositories but in doing so I realized it was sort of it was 26:33.280 --> 26:38.320 ad hoc this is an end times I'm a problem right um so I started thinking about what the minimum 26:38.320 --> 26:41.920 commonality you need is I think it's this but the implementation is a working progress 26:41.920 --> 26:48.000 do you want more examples of things that you have an included yes please yeah yeah yeah yeah yeah 26:48.000 --> 26:51.200 yeah if there's something that breaks there's it would be good to know that um so if anyone has 26:51.200 --> 26:58.560 an ideas yeah let me know you can somebody email her yeah so in maven or regular you can 26:58.560 --> 27:03.120 you can have different confide and run time dependencies you have a way to model that in the 27:03.120 --> 27:11.920 core calculus hmm so oh yeah sorry um so the question was in maven you can have different 27:11.920 --> 27:16.400 compile time and run time dependencies do you have a way to model this in the core calculus 27:16.480 --> 27:25.760 and I think the answer to that is basically the uh the versions the variables um we can have 27:26.720 --> 27:34.400 we can have something like a uh is build time variable on dependencies or is run time and then you can 27:35.040 --> 27:37.760 set that variable in the root of parameter test dependency yeah 27:38.080 --> 27:45.120 I'll do some of the ideas we have some more questions okay 27:57.680 --> 28:01.680 um 28:01.680 --> 28:06.680 reason by masking is all expected managers have their own strengths and weaknesses. 28:06.680 --> 28:07.680 Right. 28:07.680 --> 28:08.680 Right. 28:08.680 --> 28:09.680 Great. 28:09.680 --> 28:11.680 And I also want some package menus. 28:11.680 --> 28:14.680 The strictness needs is. 28:14.680 --> 28:15.680 I was just curious. 28:15.680 --> 28:16.680 Have you also thought about it? 28:16.680 --> 28:17.680 Yeah. 28:17.680 --> 28:18.680 The potential. 28:18.680 --> 28:20.680 Have you thought about it? 28:20.680 --> 28:24.680 Maybe some of the lessons from one package menu. 28:24.680 --> 28:25.680 Thank you. 28:25.680 --> 28:26.680 People would have the fact. 28:26.680 --> 28:28.680 It has when working with the Opham containers. 28:28.680 --> 28:30.680 It has been a useful way of thinking about things. 28:30.680 --> 28:36.680 Like they do a lot of bulk build testing and they were trying to figure out, you know, 28:36.680 --> 28:41.680 on a new release, what do we need to recalculate in order to figure out what dependencies 28:41.680 --> 28:42.680 we need to rebuild? 28:42.680 --> 28:46.680 And this has been a sort of useful framework for thinking about it. 28:46.680 --> 28:49.680 That's the only other use case of finding for it so far. 28:49.680 --> 28:50.680 But I am thinking about this. 28:50.680 --> 28:51.680 If you have any ideas, let me know. 28:51.680 --> 28:52.680 Yeah. 28:52.680 --> 28:55.680 Feel free to email me or message me any other questions. 28:55.680 --> 29:05.680 Thanks. 29:05.680 --> 29:10.680 Thank you. 29:10.680 --> 29:22.380 OK, thank you. 29:22.380 --> 29:26.160 Thanks you, baby. 29:26.160 --> 29:31.680 Thanks for joining us.