Share this project

Done

Share this project

Done
Developing and upstreaming complete Linux support for the Allwinner VPU, for hardware accelerated video decoding and encoding.
Developing and upstreaming complete Linux support for the Allwinner VPU, for hardware accelerated video decoding and encoding.
Created by
428 backers pledged €31,612 to help bring this project to life.

About

Allwinner VPU support in the official Linux kernel project video thumbnail
Replay with sound
Play with
sound

€31,612

428

Background

The Allwinner ARM processors are very popular in a large number of hobbyists development boards, and more and more widely used in embedded products. Support for those processors was initiated in the official Linux kernel by Bootlin (formerly Free Electrons) engineer Maxime Ripard in 2012, and has since then been improved up to the point where actual Allwinner-based products are being shipped with the official Linux kernel.

However, one key feature that is missing from the support in the official Linux kernel is support for the VPU (Video Processing Unit). This unit allows to offload to the hardware the decoding and encoding of video streams, avoiding the need to involve the CPU for such intensive operations. Having VPU support would provide a smoother video decoding/encoding experience, at a much lower CPU consumption.

A lot of existing work has already been done around supporting Allwinner’s VPU (including by Bootlin, see below), and the goal of this Kickstarter is to build on top of this work to complete VPU support and have it supported in the official Linux kernel.

It is worth mentioning that this goal is listed as one of the High Priority projects of the Free Software Foundation.

Existing work

A Wiki page on the Wiki of the Linux Sunxi community summarizes very well the existing work done around Video Engine, the code name for Allwinner's VPU.

In the Linux BSP provided by Allwinner, the VPU is supported through a closed-source binary blob. Therefore, a reverse engineering effort was started, which has produced a nice register documentation. This great effort was done by members of Sunxi community, and has allowed several open-source implementations to appear:

We believe that our approach of sunxi-cedrus is more future-proof, as it is based on a standard V4L driver, which will use a standardized and upstream acceptable user-space API. This will allow the V4L kernel driver to be accepted in the official Linux kernel, ensuring its long-term availability and maintenance.

Main goal

The main goal of this Kickstarter campaign is to continue the development of sunxi-cedrus and bring it to a fully usable state for MPEG2 and H264 decoding, supported in the official Linux kernel. This involves the following steps:

  • Making sure that the codec works on the older Allwinner SoCs that are still widely used: A10 (Cubieboard), A13 (A13-Olinuxino), A20 (Cubieboard 2, A20-Olinuxino), A33 (A33-Olinuxino, BananaPi M2-Magic), R8 (CHIP) and R16 (NES and Super NES classic). Support for the newer SoCs (H3, H5 and A64) requires more work, and is part of our first stretch goal below.
  • Polishing the existing MPEG2 decoding support to make it fully production ready.
  • Implementing H264 video decoding, since H264 is by far one of the most popular video codec.
  • Modifying the Allwinner display driver in order to be able to directly display the decoded frames instead of converting and copying those frames, which is very inefficient from a CPU consumption point of view
  • Providing a user-space library easy to integrate in the popular open-source video players
  • Upstreaming those changes to the official Linux kernel

The target of 17600 EUR of this Kickstarter campaign allows to achieve those goals.

Stretch goals

In addition to the initial Sunxi-cedrus support described as our main goal above, we have additional stretch goals to further improve Sunxi-cedrus if we receive more funding than our target:

  • Supporting the newer Allwinner SoCs, such as the H3 (Most of the Orange Pis, Nano Pi M1, ..), H5 (Orange Pi PC2, NanoPi NEO2, ...) and A64 (Pine64, BananaPi M64, ...).
    This goal requires a funding target of 22000 EUR (i.e 4400 EUR in addition to the main goal of 17600 EUR)
  • H265 video decoding support
    This goal requires a funding target of 30800 EUR (i.e 8800 EUR in addition to the main goal of 17600 EUR and the first stretch goal of 4400 EUR)
  • H264 video encoding support
    This goal requires a funding target of 44000 EUR (i.e 13200 EUR in addition to the main goal of 17600 EUR and the previous stretch goals of 4400 EUR and 8800 EUR)

Engineers for this project

Two persons will be involved in this project:

  • Maxime Ripard, working for Bootlin (formerly Free Electrons) as an embedded Linux engineer since 2011. Maxime has pioneered the support for Allwinner processors in the official Linux kernel, and is currently the co-maintainer for Allwinner processor support in both Linux and U-Boot. Maxime has written numerous drivers to support Allwinner platforms, including the DRM/KMS driver for display support. See Maxime's Linux Kernel contributions and U-Boot contributions.
  • Paul Kocialkowski, who will be working as an intern at Bootlin (formerly Free Electrons) starting March 2018. Paul has contributed to numerous open-source projects: lead developer of the Replicant project (a fully-free Android-based OS for smartphones), and contributor to projects such as U-Boot, Linux and Coreboot. See Paul's U-Boot contributions, Coreboot contributions and Linux contributions.

Delivery dates

We expect to deliver our main goals by end of June 2018, and our stretch goals by the end of December 2018.

During the entire period of development, we will be sharing the progress of the project every week in a blog post on https://www.bootlin.com.

Risks and challenges

Bootlin has already developed and released a kernel driver and associated user-space library providing MPEG2 support and partial MPEG4 support for the Allwinner VPU. We have therefore shown our ability to work on such topics. Nevertheless, there are obviously some challenges:
- There is no publicly available official datasheet for the Allwinner VPU, and all the information available has been found through reverse engineering. There is always a risk of this information to be incomplete or inaccurate, requiring tedious additional reverse engineering effort.
- The Linux kernel V4L API to support VPUs such as Allwinner's is not available in the official Linux kernel yet. Adding such an API has been in discussion in the Linux V4L for several years, but hasn't reached mainline at this point. Recent progress and discussions have been encouraging, but it might remain a challenge for this project.

Learn about accountability on Kickstarter

Questions about this project? Check out the FAQ

Support

  1. Select this reward

    Pledge €4 or more About US$ 5

    Project supporter

    Support the fully open-source Allwinner VPU project.

    Less
    Estimated delivery
    69 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge €8 or more About US$ 9

    Project supporter

    Support the fully open-source Allwinner VPU project.

    Less
    Estimated delivery
    48 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge €16 or more About US$ 19

    Project contributor

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file.

    Less
    Estimated delivery
    157 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge €32 or more About US$ 37

    Project contributor

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file.

    Less
    Estimated delivery
    70 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. Select this reward

    Pledge €64 or more About US$ 75

    Project contributor

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file.

    Less
    Estimated delivery
    29 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  6. Select this reward

    Pledge €128 or more About $149

    Project baker

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file, and receive a Bootlin t-shirt.

    Less
    Estimated delivery
    Ships to Anywhere in the world
    16 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  7. Select this reward

    Pledge €256 or more About $299

    Project baker

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file, and receive a Bootlin t-shirt.

    Less
    Estimated delivery
    Ships to Anywhere in the world
    3 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  8. Select this reward

    Pledge €512 or more About $597

    Project baker

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file, and receive a Bootlin t-shirt.

    Less
    Estimated delivery
    Ships to Anywhere in the world
    0 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  9. Reward no longer available

    Pledge €1,024 or more About $1,195

    Project baker and beer drinker

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file, receive a Bootlin t-shirt, and have the opportunity to share a drink with the project developer at a Linux conference.

    Less
    Estimated delivery
    Ships to Anywhere in the world
    Reward no longer available 5 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  10. Select this reward

    Pledge €2,048 or more About $2,389

    Project baker and beer drinker

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file, receive a Bootlin t-shirt, and have the opportunity to share a drink with the project developer at a Linux conference.

    Less
    Estimated delivery
    Ships to Anywhere in the world
    Limited 0 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  11. Select this reward

    Pledge €4,096 or more About $4,779

    Project baker and beer drinker

    Support the fully open-source Allwinner VPU project. Get your name in the project CREDITS file, receive a Bootlin t-shirt, and have the opportunity to share a drink with the project developer at a Linux conference.

    Less
    Estimated delivery
    Ships to Anywhere in the world
    Limited 3 backers
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.

Funding period

- (45 days)