Search

shirishweb

talk less, code more

Category

Programming

Python Django: Using multiple database, inspectdb and admin for existing database

One of the most powerful parts of Django is the automatic admin interface. It allows all users including developers to manipulated data inside database table. Often people use django admin tool to manage their database rather than using the console management tool provided by the database software itself or any other tool which are very limited and specific to one type of database only.

I have an existing setup mysql database with all the required schema and stored data. I want to use Django’s powerful administration tool to manage some of the stuffs on the database.

Lets get started on.

I have setup a django project called boad with necessary configurations updated in the settings.py file. I have added my mysql server as default database for the project.

Since running migration using django management command would create default django tables like django_sessions, django_migrations etc on my database which I dont like as I would like to keep my database very neat and clean as it was prior to django setup. So I decided to use django multiple database feature using database router that would route database operations of all other django related table to other database. I have setup a database router which will route read,write and syncdb operation of all tables not related to my existing database.

First in my settings.py I set up two databases as:

DATABASES = {
    'sqlite': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    },
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'HOST': 'localhost'
        'USER': 'root',
        'PASSWORD': '',
        'NAME': 'boad_database'
    }
}

DATABASE_ROUTERS = ['boad.router.NonBoadAttributeRouter',] #Router's module path

I have used two database sqlite and mysql (default). I would be using sqlite to store data that are specific to django tables.

I created a router.py file in the project module directory as:

class NonBoadAttributeRouter:

    non_boad_attribute_tables = ['auth', 'admin', 'contenttypes', 'sessions', 'messages', 'staticfiles', 'migrations']
    def db_for_read(self, model, **hints):
        if model._meta.app_label in self.non_boad_attribute_tables:
            return 'sqlite'
        return None

    def db_for_write(self, model, **hints):
        if model._meta.app_label in self.non_boad_attribute_tables:
            return 'sqlite'
        return None

    def allow_relation(self, obj1, obj2, **hints):
        if obj1._meta.app_label in self.non_boad_attribute_tables or obj1._meta.app_label in self.non_boad_attribute_tables:
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if app_label in self.non_boad_attribute_tables:
            return db=='sqlite'
        return None

Here I have stated that if application label is either of labels defined in non_boad_attribute_tables python variable, they should operate by “sqlite” database. Each router’s function would check for app label and route to “sqlite” database as per required or would return “None” which would tell django to fall back to “default” database for database operation.

Django have an “inspectdb” management command which is very useful to generate Django model from the existing database. As an example here I have use inspectdb to generate model for my course_category table in mysql.

venv) D:\Learnings\Django\boad\src>python manage.py inspectdb course_category
# This is an auto-generated Django model module.
# You'll have to do the following manually to clean this up:
# * Rearrange models' order
# * Make sure each model has one field with primary_key=True
# * Make sure each ForeignKey has `on_delete` set to the desired behavior.
# * Remove `managed = False` lines if you wish to allow Django to create, modify, and delete the table
# Feel free to rename the models, but don't rename db_table values or field names.
from __future__ import unicode_literals

from django.db import models


class CourseCategory(models.Model):
 course_cat_id = models.AutoField(primary_key=True)
 coursename = models.CharField(max_length=100)
 code = models.CharField(max_length=10, blank=True, null=True)
 project = models.CharField(max_length=30, blank=True, null=True)
 created_by = models.CharField(max_length=60)
 created_date = models.DateTimeField()
 deleted_by = models.CharField(max_length=60)
 deleted_date = models.DateTimeField(blank=True, null=True)
 deleted = models.IntegerField()

class Meta:
 managed = False
 db_table = 'course_category'

I have generated an model from existing database. I can do same for other mysql tables and create put it inside their own respective django app created using

python manage.py startapp courses

app_str

Now finally in admin.py file of each django app folder, I have register each model in admin as:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.contrib import admin

from course.models import CourseCategory

class CourseCategoryAdmin(admin.ModelAdmin):
    list_display = ['course_cat_id', 'coursename', 'code', 'project']
    list_filter = ['project',]
    list_editable = ['code',]

admin.site.register(CourseCategory, CourseCategoryAdmin)

 

Running django’s migration command wont create any other tables in my mysql database but instead will be created in other sqlite database.

After successful migration I can test my django app using:

python manage.py runserver

course_admin_interfaceI can now successfully setup admin interface for managing my mysql database.

This approach of creating admin interface is applicable to most of popular database systems like Oracle, postgres etc. Django is going more popular day by day and equally getting more recognition for its quick and powerful admin interface generation.

Python defaultdict() versus dict.get()

Python is a great language. Dict is one of data structure available in python which allows data to store in the form of key/value pair. Many times we might encounter a situation where we have to retrieve a value using the dictionary key. When our key is not found in the dictionary it will throw an exception of KeyError.

KeyError

The solution to this is using defaultdict or dict.get() to return a default value if the key is not found in the dictionary.

Lets dig into both of these

defaultdict can be found in collections module of python. So, in order to use it, we have to import it first as:

from collections import defaultdict
mydict = defaultdict(int)

defaultdict constructor takes default_factory as argument which is a callable. This can be

int : default will be integer value of 0

str : default will be empty string ”

list : default will be empty list []

and so on.

If we want our own default value then we can pass function pointer.

Lets say that we want default value to string ‘default’. We can achieve this in defaultdict as

def mydefault():
        return 'default'

mydict = defaultdict(mydefault)
print mydict['test']

will output ‘default

The same result can also be achieved using the dict.get method as

mydict = {}
mydict.get('test','default')

will output ‘default

Thus same result can be achieved using both approach. dict.get have to provide a default value every time it is called whereas using defaultdict we cant setup a default value only one time.

Now lets check the efficiency of both of them in term of execution time.

For this we will be using ipython notebook since it has got %timeit command to measure the execution of any python statements over any desired looping of same statements.

In Ipython notebook environment, let us create two function that implements defaultdict in one and dict.get method in another.

defining_two_defs

Here we have implemented a() function to execute defaultdict value retrieving and b() to use dict.get method. Now lets calculate the execution time of both function in 100 loops as

defaultdict_timeit

dict.get_timeit_2

First executing a() function over 100 loops we can see that it took over 8.88 micro sec per loop using the best of three.

dict.get_timeit

Here when we run b() function over 100 loops we can see that it tooks 19.3 micro sec per loop using the best of 3 approach.

Hence defaultdict seems more efficient over dict.get method.

Lets rerun this test again but this time will be using default %timeit command of notebook which will be looping the statement for default 1000 loops

defaultdict_timeit_2

Clearly here also defaultdict seems more efficient that dict.get method and the experiment shows that defaultdict more that two times faster than dict.get method.

Javascript Module: Basic Desing and Pattern

1. Creating a Module
First we need to understand about “Immediately Invoked Function Expression (IIFE)”. These IIFEs are simple function when declared, are called immediately.

IIFE

This will also create a new scope where we can put all out logics. This is an anonymous module. We can namespace this module as

Simple Module

2. Creating a private method

Private method simply is a method that are wrapped inside a module, intended to prevent access from externally outside of the module. This will make our methods more secure. Private methods are useful when we have to access sensitive data over the internet or make external server calls.

code3

3. Module Gateway: “Return”

Inside the module, we will reveal only part of the scope for external access. This is made possible by using the return statement within the Module. This return statement return object the mode and is made accessible from the module’s namespace.

code4

We can access this public method as:

code5

4. Code management with “Locally scoped object literals”

We see in above code that we aren’t namespacing our return object. We can also namespace our returned object with the module scope that makes our code more managed and readable. We can also segregate our private methods from return within the namespaced object.

code6

5. Desing better with revealing module pattern

With revealing module pattern we only return things that are necessary for external access for our module to work perfectly as intended. This create a public gateway to our module and get access to only this that we want to reveal.

code7

6. Extending our module

There might be some cases that our built method doesn’t work fully and lack some of the functionality needed on some cases specifically. In that case we can easily extend our pre built module and add some other functionality within it.

code8

Here we have extended our Module to NewModule that have new “newMethod” added methods.

NewModule has an argument Module || {}. Here we have passed pre-built Module to the NewModule. If this Module is not defined or “undefined” then we pass new object “{}” as argument to our NewModule and extend it.

Create a free website or blog at WordPress.com.

Up ↑