Share this project


Share this project

A free software fixed media format for HD video, based on open standards, with metadata and localization but no DRM restrictions
A free software fixed media format for HD video, based on open standards, with metadata and localization but no DRM restrictions
467 backers pledged $21,421 to help bring this project to life.

Not my proudest moment as a programmer, but...


All right. I have, I believe, resolved the nasty bug I was dealing with, and now understand how to do event callbacks with Rocket. The API for this is seriously twisted, and I could rant about how bad the design is, but the important thing is that I am able to use it anyway.

Still, I suppose I'll rant a little anyway.

The API works by pushing the context variables from the page context in which the call was made into the global namespace. There's two variables: "event" (which contains the data from the event that happened) and "self" (which contains the Rocket "RML Element" object which generated the call. There's supposed to be a third variable, "document", but my tests didn't show it being defined (maybe it's not always available? I'm not sure).

You read that right, Python programmers: the call sticks an object named "self" into the global namespace.

If the callback is a method, as happens to be necessary in my use case, then you have to use a non-standard method call with some other name for "self" and grab the "element" from the global namespace.

What happens next is a style decision, and it's not clear to me which is clearer. Either you can just keep using the non-standard name for "self" (that is, the name for the parent object of the method). Or you can delete the global "self" and rename the local variable back to "self" so that the rest of the method works. (The snippet below uses the latter solution, but I don't know if I'll keep it that way or not).

It's ugly as hell, and such an API erodes my confidence in this package a bit. But I think (and hope) the rest of the package is more sensible.

Wow. I did not expect to need this.
Wow. I did not expect to need this.

Anyway, it DOES work.

It also does work even when VLC is playing video -- I wasn't sure whether that would work or if I'd have to implement a listener through VLC as well. So that's a relief.

I'll need to do a little clean up of all the introspection and instrumentation code I added to figure out what was going on. But after that, I should be able to move onto implementing menus correctly.

So... yay!

Status: Bug Hunting


It's been awhile since I posted anything, and I had expected to have the player more finished by now, but I ran into some bugs with the underlying C++ libraries (or one bug from there and one I haven't isolated yet, but I suspect will turn out to be similar).

The first bug, in the Rocket Standalone module, took some time to ferret out. It turns out you can have some really subtle Python bugs when the module isn't actually written in Python -- this one was a reference-counting error which resulted in executing a random function as a callback (just whatever happened to be in that memory location).

And now I'm stuck on one with the event handling system in the Rocket library proper.

For some reason I'm getting a  "Keyerror" traceback which is supposed to mean that Python is attempting to do a dictionary look-up and not finding anything, but the weird thing is that the line it reports the error in doesn't even mention a dictionary look-up. Most likely this is another case of something not being done quite right in the C++ module, and it'll either have to be patched, or I'll have to find a work-around to avoid triggering it.

If this sounds trivial, well it does to me too, but it still blocks me from moving forward, as I really need a working event system to do testing of the menu system.

Right now, I just need to isolate the bug, and I think it's going to take several hours of careful testing to find it, since the traceback is apparently not going to be very helpful. I expect to block some time next Tuesday-Wednesday to do that. Really hoping that'll be enough time to find it.

Somewhat related (in that it's the reason I'm having to deal with this problem now): There's a small error in the architecture diagram I drew earlier. It turns out I can't just use the Python standard library to do the keyboard input, because it depends on the window focus. Instead, I'm using Rocket's event system to monitor the keyboard. I'm not sure yet whether it will be possible to use that while VLC is running or if I will have to switch back and forth depending on the player mode. I'm prepared for either case, though.

I'll post again once I figure out what's going on with that. Thanks for your enormous patience, once again.

Rocket/VLC-Standalone Python Extension Module Completed


Last year, I announced the decision to use the Rocket library for menus rather than WebKit, primarily for security reasons, although it's also worthwhile to make the package simpler to write and easier to read -- which is important if it's to do a good job as a reference implementation.

Rocket solves these problems by being a much simpler, thinner interface layer (it really is just the renderer -- actually following links will be handled by callbacks to my code, which allows me to completely control the browsing process, making it both easier to write and more secure).

What I did not mention in my previous post (because I did not realize it until I started running tests in early December), was that Rocket is actually not a complete Python extension module. This is not really clear from the documentation (I may have to do something about that!), and I just figured I was having some trouble initializing it correctly (after all, the demos ran fine). Well, it turns out that, for me, "Rocket is only half a rocket-ship". The other half has to be provided by a backend rendering library -- something like OpenGL or SDL2. And that work has to be done in C++, you can't fix it from Python.

So I needed to get the other half of the rocketship built.

I was stuck on that for awhile, but ended up commissioning Tom Mason, who is the author of "Freeablo" (a free-software alternative to the Diablo game engine), and who therefore has C++ experience with Rocket and the SDL2 backend, which is what I wanted to use.

He released this to me just last week, and I've been running tests with it. Everything seems to be working exactly as it should, and it even works in the same logical X Window as VLC, so integration of menus and playback is very smooth and quick.

So this part is now done.

I wish I could do justice to how much better and easier this makes my situation!

There is still quite a bit of Python code to write, but it is much simpler, more familiar to me, and all focused on getting the desired Lib-Ray player behavior.

This will become a new "ichinose" branch in the SVN, as soon as I get the source organized. Here's a block diagram of this new Rocket-based architecture:

Rocket-Based Lib-Ray Player Architecture
Rocket-Based Lib-Ray Player Architecture

The orange "Rocket-Standalone" is what Tom Mason just completed, which allows all of the C++ stack to fit together. The Python layer above that are existing Python bindings for these libraries. The Lib-Ray Player Python application sits on top of that -- and that's what I'll be working on over the next month. Some pieces of this, like the Volume Object Model and the Localization code can be used with little or no re-writing. The "Menu-Browser" and "Video" modules will pretty much have to be completely re-written (although they will now be a lot simpler and easier to understand).

I will add support for infrared remote controls (probably based on "liRC") as well as mounting and automounting the UDF volume file-systems after the basic stack is all working with pre-mounted volumes and keyboard/mouse control.

I guess the stickiest part now is that I'm going to have to do some translation to go from a more standard XHTML+CSS markup on the Lib-Ray volume to the RML+RCSS markup that Rocket understands. This is mostly a subset, which I could just declare to be standard for Lib-Ray. But there are some oddities that need to be seen to:

1) Background images are handled in a non-standard and more complex way. This is undesirable since Lib-Ray menus will probably use them extensively. So I'll need to fix this in the filtering process.

2) Rocket doesn't actually understand hyperlink ( A-HREF ) tags at all, it just allows you to set event handlers on any element. So my filter will need to find the tags, read the URLs, and assign handlers.

Those are simple, easy-to-test text-filtering tasks. Very familiar territory.

After that, I'll need to implement hyperlink navigation, including dispatching my special URLs for sending messages to the VLC video player. Of course, this fine control over navigation was one of the main reasons to choose Rocket.

I'm hesitant to make projections because of all the ones I've made that have fallen through on this project, but I really do think I'll be done with the player by the end of March now.

Menu Internationalization Testing


I've completed the primary work on internationalization tests for Lib-Ray. After much searching for a better alternative, this is a combination of using ElementTree and simple regular-expression-based corrections to the output. The result works very neatly, as you can see (these are from the test example included in the module -- I'm not quite sure if I can or should try to convert this to a self-checking unit test or not).

BTW, the translations are mostly by me at this point, using online resources, and I obviously don't speak all of the 12 key "language system" languages that will be included in all Lib-Ray players. So I will be soliciting corrections from native speakers sometime soon!

The mechanism is very simple. The most important thing to internationalize in Lib-Ray are the functional button labels. This context allows for unambiguous vocabulary-based translation. An attribute is added to an HTML element, and if recognized, the text within the element is replaced with the localized text. At present, the attribute is "label", although I'm reviewing whether that's the best choice in terms of HTML and XML standards (other possibilities: "id", "libray:label", etc).

One nice thing about this is that it really is not dependent on the author of the release to do the internationalization. The button text is changed by the player itself, so the internationalization and localization is just done there. As long as you have a language pack for your locale installed, all Lib-Ray volumes will have their buttons correct for your locale.

There is one issue for authors to consider: the length of words may be quite different. You can see here that the Russian and Chinese for "PLAY" differ by a lot in size on screen, for example. This will be something to take into account in designing your menus.

A secondary mechanism for authors to provide alternate text for different languages is yet to be developed, but should be possible using the same ElementTree-based manipulation of the source menus (and I think there is already a standard way to do this in HTML, so we'll just honor that, if that's true). However, I'll probably defer that until after we have a working player released.

Testing Menu Localization
Testing Menu Localization

Switching the Lib-Ray HTML/CSS Menu Rendering Engine


After much deliberation and quite honestly, a bit of quiet whimpering, I have made the decision to remove WebKit from the menu part of the player, and replace it with a different solution based (probably) on librocket. This has been bugging me for a really long time, and I think it has to be done.

Although I have just about got it working, I have realized that I really am afraid to release it. There's way too many potential security holes introduced by using a full-fledged browser engine for our menus, because a modern browser can generate calls out to the internet in so many different ways. And I'm not likely to feel any better about it later, which is the thing I had to work around to. I've been thinking this could be a supremely stupid thing to do. Lib-Ray has enough PR trouble from running this late, not to introduce more by making it into an infamous malware vector!

Sometimes, I have realized, those nagging doubts are telling you something real, and you should listen to them.

The Lib-Ray menus are supposed to be a "safe" environment, so you shouldn't have to worry about exploits that reach out to the web from your movie's player menus. I knew this, but I was very worried about the extra difficulty of re-developing the HMTL rendering capabilities, as I feared the alternative was to develop the renderer from scratch (or try to extract it from WebKit, which would be no small task for me).

However, I'm now investigating a library called "librocket", which provides HTML/CSS-based rendering, but only that -- it was originally targeted at producing user interfaces for games. It will take some evaluation to decide for certain that it can do all of the things that Lib-Ray requires, but the more I look at it, the more I think this is a much better deal than the mass of work-arounds that is currently my WebKit-based solution for menus.

Librocket is an HTML/CSS based GUI library targeted mostly at games, and looks like a better option for Lib-Ray menus.
Librocket is an HTML/CSS based GUI library targeted mostly at games, and looks like a better option for Lib-Ray menus.

This also means a switch from an "opt out" approach to network security to an "opt in" approach. In other words, any calls made to the internet will have to be added by me, rather than me having to patch every possible unwanted connection and behavior. Metaphorically, using WebKit is like taking a basket and trying to tape up all of the holes, whereas using librocket will be like taking a metal bucket and punching any holes I find I need. This is a MUCH safer approach.

I'm a little sad to be throwing away months of tinkering with WebKit, and I might not ditch it entirely. It could for example be a good way to handle the unsecured "Extras" parts of volumes or for actual web-browsing (easier than implementing a browser back-end for librocket, certainly). But using different software for menu-browsing versus web-browsing creates a much better firewall between the two activities. So that's probably a much better approach.

There are a few tradeoffs with librocket. It has not been available as free software for as long (re-released under MIT license, 2010), and so distribution packages are not available. Thus, I will probably have to provide packages for it myself as well as build it from source for development. It expects menus to be defined in an XML-compatible syntax, which means a partial move to a more XHTML-based format for Lib-Ray menus (not actually sure this is a bad thing, especially given the acceptance of DRM video into HTML5, but it's a change from the proposal). I'm not sure yet how limiting that is likely to be.

However, the librocket library also has other advantages besides security. The documentation is much better, especially for the Python bindings (because those are basically non-existent for WebKit). The code will be much simpler, as I can now get rid of the horrendous "fake client-server architecture". Librocket has a simple callback architecture, like most GUI libraries. That's a big advantage, since the point of this code is largely to be read by other developers who want to create new implementations. Since it was designed to overlay menus on game graphics, it may make it feasible to create animated menus and other effects that are awkward with WebKit.

As for the impact on delivery, I actually think a complete librocket implementation is going to take less time than making the WebKit version safe enough to release. Certainly it will allow me to concentrate on what should be the top priority, which is defining the allowed menu language -- something that has been dragging considerably because of the relative inflexibility of the WebKit approach. I know this has been a long time in coming already and it's long overdue, but I strongly feel that the security and quality concerns have to come first.