Share this project

Done

Share this project

Done
Supporting a wider range of awesome PostgreSQL features in Django - including hstore, JSON and full text search
Supporting a wider range of awesome PostgreSQL features in Django - including hstore, JSON and full text search
334 backers pledged £14,411 to help bring this project to life.

About this project

£14,411

334

Background

Django is a database-agnostic web framework. This means that the database fields provided by Django are designed to work across SQLite, Oracle, MySQL and PostgreSQL. In fact they also work on several third-party database backends. However, the majority of applications built using Django are not, and do not need to be, database agnostic.

PostgreSQL is arguably the most popular database within the Django community and it certainly has the widest feature set of the core supported databases. PostgreSQL's advanced features have been the subject of many talks at Django conferences and events, and there are a handful of libraries in the community for using them. Historically Django has made it very hard to implement well-designed custom fields. This was in part due to a limited field API, which provided no way to do custom lookups. Whilst this worked fine for a UUID field in PostgreSQL, it meant that the implementations for arrays, hstore, JSON etc. either lacked some nice features or had unpleasant implementation details. For a developer, it represents a sometimes bewildering selection of libraries with their own distinct quirks and flaws.

I want to make things better. There should be one obvious way to use these data types and they should be a core supported feature of Django.

The project

I am aiming to write a new contrib module for Django: django.contrib.postgres. The key features will be:

Support for many new PostgreSQL-specific data types, including:

  • hstore
  • json
  • array
  • enum
  • interval (maps to a timedelta)
  • int4range, tsrange and other range fields
  • uuid

In addition, there will be support for using tsvector-based full text searching on appropriate fields. These new features will of course be fully tested and documented.

Any additional funding above the initial £2500 will go towards more features.

  • £5000 - Improved date-based lookups for all databases. This means it will be possible to do lookups like field__year__lte. It will also include other time filters for PostgreSQL, such as __decade and __century.
  • £7500 - PostgreSQL-specific custom lookups for existing fields, including string and mathematical operations. The new PostgreSQL-specific data types will also now benefit from a wider range of custom lookups.
  • £10000 - Custom indexes, including different index types (e.g. gin indexes) and functional indexes (e.g. indexing on year of a date field).
  • £14000 - Support for views (and materialized views) in PostgreSQL and other database backends.

Additional funding over the £14000 goal will go towards improving the support for custom transforms throughout Django, refactoring contrib.gis to remove the need for custom managers, and helping with release blockers for future releases. Some funding may also be given to help compensate those who do a lot of review work on the project.

What about $MYDATABASE?

The plan is to implement contrib.postgres without hackery. It will not be required to change the database backend or use a custom manager. This will mean some refactoring of ORM code, including some refinement of the new Lookup and Transform classes for providing custom lookups. It will make implementing a custom database field much easier on any database, as well as providing a lot of reference examples for the PostgreSQL types. The choice of PostgreSQL comes down to a combination of personal preference and the existence of a wider range of missing features than any other database Django supports.

About me

I'm a Django core committer, a full-time parent and part-time freelance developer. I have time around my parenting commitments to work on Django, and the enthusiasm to code in the evenings and at weekends, which comes from not working at a computer all day.

I love PostgreSQL and have used it as my main database for three years, often wishing that it were easier to use its more advanced features. Whilst there are libraries which have met my use cases, the code has always felt ugly. We can do better.

The idea of contrib.postgres has been in my head for about a year. I've been researching how the ORM works and helping with the implementation of custom lookups. This means I've got a fairly clear idea how to implement the features I'm proposing.

Why Kickstarter?

I'm a full time parent, but I have to earn some money. This is a project of significant size, and I would not be able to dedicate the necessary time to it without funding, as I would need to take on more paid freelance work instead. Kickstarter was very successful for the new migrations framework, and I feel that being able to guarantee a significant amount of time to work properly on a project like this is the best way to ensure the end result is of the highest quality.

The kickstarter is my own - not officially organised by the Django core team or the Django Software Foundation. However, both the DSF and the core team have given their approval for the project.

The Django name and logo are trademarks of the Django Software Foundation and are used here with permission.

When will it be released?

The project is very modular. You could say that it is in fact 20 or more smaller projects, with each data type being a feature on its own. This means that each part can be reviewed and merged individually. Work will start towards the 1.8 release of Django which should see most of the simpler fields. Full text search and some more advanced cases like the enum data type will hopefully land for 1.9. Stretch goals may take longer.

Where is my money going?

The amount raised will guarantee at least 80 hours of work on the project. This will include the implementation of model fields and form fields, full unit testing and a lot of documentation with examples.

Risks and challenges

Software development always carries some element of risk. Unexpected edge cases can appear and good design can turn in to complicated implementation. However I'm confident these issues can be overcome.

There is also the risk that the patches would not get merged due to lack of review. Quite a number of the core team have expressed that they are very keen to see these features land and have use cases in their own projects, so there is significant interest in seeing the patches land. One of the developers of django-hstore has offered to help with review, and you can help too!

Learn about accountability on Kickstarter

FAQ

Have a question? If the info above doesn't help, you can ask the project creator directly.

Ask a question

Support this project

  1. Select this reward

    Pledge £5 or more About $8

    Thanks for supporting the project!

    Less
    Estimated delivery
    120 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge £15 or more About $25

    Loads of thanks! Your support is hugely appreciated.

    Less
    Estimated delivery
    75 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge £40 or more About $66

    All the thanks I can muster and credit as a supporter of the project on the project blog.

    Less
    Estimated delivery
    50 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge £100 or more About $164

    Your credit for the project will include a logo and a link to your website.

    Less
    Estimated delivery
    42 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. Select this reward

    Pledge £300 or more About $492

    Detailed email support on using the new features, when you should or shouldn't use them, and upgrading to them.

    Less
    Estimated delivery
    Limited 5 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  6. Select this reward

    Pledge £1,000 or more About $1,641

    You, or your company, get a full day of remote or on-site support (travel not included). You also get email support, and lots of credit in talks and on the project blog.

    Less
    Estimated delivery
    Limited 3 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.

Funding period

- (31 days)