Share this project

Done

Share this project

Done
Full-feature package for easy importing and exporting data to avoid having to "reinventing the wheel"
Full-feature package for easy importing and exporting data to avoid having to "reinventing the wheel"
23 backers pledged $900 to help bring this project to life.

About

Django opensource improved import-export package

$900

23

Update: first working prototype

Motivation. Why yet another package?

In the process of developing small packages for exporting and importing data for price aggregators, news portals, shops and other sites, we are forced to do repetitive tasks. To solve this problem we generally use a ready-made solution in order to avoid having to ”reinvent the wheel.”

After analyzing existing packages (see table below), we identified 4 packages in total and only 1 of which is active and usable: django-import-export. We did not include django-tablib in our analysis, as it's a mirror of django-import-export without the admin interface. After using django-import-export in an actual production project, we came to the conclusion that its functionality and flexibility is clearly not sufficient for normal exporting and importing data especially when we have to reinvent the same thing over and over despite the fact that we have a code base and previous experience.

The aim of this project is to write a generic package for importing and exporting data, which will save you from repetitive work and will facilitate the data exchange in your project.

Another goal of this project is to create tutorial about developing this package in live mode on our site http://mtr.website. So everyone can join in to development process and share ideas about project.

Features:

  • Multilingual (i18n)

  • Uses Celery for background tasks and for processing large volumes of data

  • Creates reports about importing and exporting operations

  • Model API - registers models for use in package

  • Field settings:

    • maps model fields with data fields

    • adds and removes fields (ability to skip fields on import)

    • includes drag&drop sorting fields

    • related models(fields) import-export by choosing main model
  • Supports: CSV(native), XLS (using: xlwt-future, xlrd), XLSX (using: openpyxl optimized writer, reader mode, for fast processing of large volumes of data)

  • Saves import templates for the processing of data from various sources and for simplicity

  • Integration with standart django admin app

  • Supports: Django 1.6+ Python 2.7, 3.3+.

Github repository: https://github.com/mtrgroup/django-mtr-sync

Additional features

Any additional funding will be allocated to the development of additional features, including the following:

> $500:

  • Adapter API for supporting other formats

  • Adding support of JSON, YAML, XML, ODS

  • Data range settings (start, end cells in table), for example, if you need to import data where there is a header with logo or any other unnecessary information

> $750:

  • More tests and fixing bugs as soon as possible

  • Permission control for import settings using django auth, to minimize human errors. For example, this would allow only the manager to choose the settings template for import and to upload files without configuring

  • Export templates for (XLS, XLSX, ODS)

    • upload custom templates

    • set start cell of exporting data

> $1000:

  • Improved documentation

  • Filtration

    • Filter API — write own filters

    • Create filter from admin panel using embedded django template language, for example:

      • {% if field|is_number %}{{ field/2 }}{% endif %} (pseudo code)

    • Standard filters

      • If object exists in database and does not exist in import data, then delete it

      • If object exists in database and does not exist in import data, then set object parameter to whatever you want

      • Create object if it doesn't exist in database with current parameters from import

      • Assign object to main model field (ForeignKey, ManyToManyField), for example, category or tags separated by coma

> $1250:

  • Template integration with django-grapelli, django-suit

  • Periodic export for automatic updates

  • Email notification 

> $1500

  • Video tutorial how to set up package and use it

  • Plugins for editors ckeditor and redactor — insert file from already exported files

How will it look like?

In order to have a clear idea of how it will look and work, please see this mockup below (this is not the final product):

What license will be used?

For your convenience, the package will use an MIT license, which you can modify and use in your project any way you like.

About us

We are a group of enthusiasts who have been using Python and Django since version 1.0. If you have any questions, you can ask them here or send an email to inboxmtr@gmail.com or visit our website: http://mtr.website.

Why we work «for food»?

We believe that this open source package will help prevent you from “reinventing the wheel” in your project. Certainly, this task has been completed by our colleagues more than once. We really want to implement the project because we need it, and of course because it will be useful for all. That’s why we want money only for life’s necessities (existence).

Why we are here?

Kickstarter is the best way to get an independent assessment and the opportunity to listen to your opinion. It also allows us to assess the demand for the development of necessary packages, as well as make it easier for us, the developers, to increase your efficiency.

How we spend money?

We plan to use money to pay for food and office rent. Of course if any money will remain, it will be used as payment for work.

Risks and challenges

Our approach to the operation allows us to minimize risks while allowing us to complete the project as described. We minimize extraneous factors and maximize our focus on the task.

The entire package will be implemented within 2 months taking into consideration any risks and unforeseen circumstances.

Our task is to make a high quality product. You can watch the whole process through the github repository or kickstarter, as well as participate in testing. We will be continuously reporting our progress.

Learn about accountability on Kickstarter

Questions about this project? Check out the FAQ

Support

  1. Select this reward

    Pledge US$ 5 or more About US$ 5

    Thanks!

    Less
    Estimated delivery
    6 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge US$ 10 or more About US$ 10

    Thanks and kudos to you!

    Less
    Estimated delivery
    5 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge US$ 25 or more About US$ 25

    Big-big Thanks! We appreciate it!

    Less
    Estimated delivery
    4 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge US$ 50 or more About US$ 50

    A huge Thanks to you for funding the project. We really appreciate your support.

    Less
    Estimated delivery
    4 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. Select this reward

    Pledge US$ 100 or more About US$ 100

    Our gratitude has no limit! Your name or company name will be added to the list of project credits.

    Less
    Estimated delivery
    0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  6. Select this reward

    Pledge US$ 250 or more About US$ 250

    You are a generous philanthropist! Your name or company name with link and logo will be added to the list of project credits.

    Less
    Estimated delivery
    2 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.

Funding period

- (14 days)