Cassandra and Django: Building the front end for TikTokTweet

Hey there, I’m back again to talk about my latest project, TikTokTweet, a tweet scheduler for python. This is the third blog post the series dealing with the project, if you need to catch up, go back and have a look at the first blog post in the series. Today I’m going to get into a lot more details on what I’ve been up to. I went over the models in my last blog post, they have changed a bit since. Just as an overview, these models were derived from my original CQL schema:

/* Table to store tweets.
 * There is timestamp for the initial creation of the tweet and one for date/time
 * modified incase any edits are made.
 */
class Tweets(Model):
    id = columns.UUID(primary_key=True)
    tweet = columns.Text()
    description = columns.Text()
    created = columns.DateTime()
    modified = columns.DateTime()

/* Table to store the different queues created.
 * A queue is a list of tweets of a
 * particular category. Each queue type gets its own unique id.
 */
class Queue(Model):
    id = columns.UUID(primary_key=True)
    name = columns.Text()
    created = columns.DateTime()
    modified = columns.DateTime()

/* Table showing what tweets are associated
 * with which queues and time for the tweet to be sent out.
 * Each tweet is in a particular queue with a certain set
 * time to be set out, hence the chosen primary key.
 * Clustering order for time_to_send is descending
 * such that next tweet to go out is at the top of the list.
 */
class Tweets_queue(Model):
    queue_id = columns.UUID(primary_key=True)
    time_to_send = columns.DateTime(primary_key=True, clustering_order="DESC")
    tweet_id = columns.UUID(primary_key=True, clustering_order="ASC")

/* Tables to hold responses to tweets.
 * TIMEUUID is used as a distinct identifier here
 * to deal with the unlikely possibility that two
 * people could reply to a tweet at the exact same time.
 * The queue_id is including in the primary key such that
 * we can select the responses for a particular queue.
 * Ordering by time descending is chosen such that the l
 * latest reply will be at the top of the list.
 */
class Queue_tweet_responses(Model):
    queue_id = columns.UUID(primary_key=True)
    time_received = columns.DateTime(primary_key=True, clustering_order="DESC")
    user = columns.Text(primary_key=True)
    tweet_id = columns.UUID()
    response = columns.Text()

/* Table to hold tweets which have already been sent.
 * This is so we can keep track of tweets that have already
 * been sent out. Eventually this could be work to help us
 * if the program crashes and we don't end up sending the
 * same tweets out again by accident.
 */
class Tweets_sent(Model):
    queue_id = columns.UUID(primary_key=True)
    time_sent = columns.DateTime(primary_key=True, clustering_order="DESC")
    tweet_id = columns.UUID()

/* Table to hold Twitter ids which have already been sent through 
 * Twitter. This id is can be obtained once the tweet is sent out.
 */
class Tweets_sent_by_twitter_id(Model):
    twitter_id = columns.BigInt(primary_key=True)
    queue_id = columns.UUID()
    tweet_id = columns.UUID()

Queue_tweet_responses has a new column ‘users’, so I can keep track of who the replies are from. Also, a new table, tweets_sent_by_twitter_id has been added.
Now, I wanted to go through how to set up Cassandra with Django (and enable the use of cqlengine along with it) in one shot. Django comes with SQLite by default, and some tinkering needs to be done to connect Cassandra up as the database. Using Django Cassandra Engine, a database wrapper for Django that uses cqlengine, this becomes pretty quick and painless. I first installed Django Cassandra Engine using PyPi:

pip install django-cassandra-engine

After that, I went into my settings.py file. Here I see all the default stuff Django had set up for SQLite. I made these changes to the settings.py file so that my app would be ready to use Cassandra. I added Django Cassandra Engine to the top of my INSTALLED_APPS.(It should be on top.)


INSTALLED_APPS = (
'django_cassandra_engine',
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'tiktokadmin',

)

Next I changed my DATABASES setting. I keep the default as SQLite though and add Cassandra as an option. ‘NAME’ will be the name of your keyspace. I did not add Cassandra as the default database, as Django will be using SQLite for administrative functions in the backend (i.e. user sessions). It is possible that Cassandra could be used for such a purpose, but I will not be addressing that at this time.


DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
'cassandra': {
'ENGINE': 'django_cassandra_engine',
'NAME': 'tiktok',
'TEST_NAME': 'test_db',
'HOST': '127.0.0.1',
'OPTIONS': {
'replication': {
'strategy_class': 'SimpleStrategy',
'replication_factor': 1
}
}
}
}

I have my models based on my schema in my models.py file. Make sure your Cassandra instance is running. From the command line, run the command:

./manage.py sync_cassandra

You could be able to see the tables derived from the models being created in your Cassandra database. You will need to run this every time you make changes to the models in your models.py file, so that the changes can synced with the database.

views.py

Now I was ready to put some stuff in views.py. Views are a fundamental part of the Django framework. A view is a Python function that will take a Web request and return a Web response. It contains whatever arbitrary logic is needed to return that response. For example, supposing we wanted to create a tweet:

def create(request):
    template = loader.get_template('tiktokadmin/create.html')
    queue_list = Queue.objects.all()
    context = RequestContext(request, {'queue_list': queue_list})
    return HttpResponse(template.render(context))

Here is the ‘template’:

Screen Shot 2015-02-12 at 7.10.17 PM

Screen Shot 2015-02-12 at 7.25.56 PM

The create view provides the necessary data from the database needed to make this part of the web app happen. The list of queue names is pulled from the queue table in the database and displayed as a list on the webpage, where the user gets to make a selection when they create their tweet as to which queue to add the tweet to. We get the ‘template’ from actually grabbing the associated HTML file we’ve constructed to give the layout the user see at this particular part of our webapp. These can be found inside the ‘templates/tiktokadmin’ folder.

This code loads the template called 'tiktokadmin/create.html'

template = loader.get_template('tiktokadmin/create.html')

When you use a template in Django, it is compiled only once. For optimization, it is stored for future use. A template can have variable names, such as queue_list in the code snippet below from ‘tiktokadmin/create.html’.

...
<label>Add to queue:</label>
<select name="queue_id">
<option value="0">Do not add to queue</option>{% for queue in queue_list %}
<option value="{{queue.id}}">{{queue.name}}</option>{% endfor %}
</select>
...

For this template to work, its going to need the list of queues available from the queue table. We make a CQLEngine query to retrieve all the objects from the queue table. This can be accomplished with the .all() method, which will return a QuerySet of all objects in a table. We obtain all Queue objects:

queue_list = Queue.objects.all()

Now the template we loaded earlier gets context passed to it. A context is a is a dictionary mapping template variable names to Python objects. This is a dictionary with variable names as the “key” and their values as the “value”In this case, it is the queue_list we obtained. ‘queue_list’ is the key, and the queue_list obtain from the code above is the value.

context = RequestContext(request, {'queue_list': queue_list})

We now return a HTTPResponse object with the result of the rendered template.

return HttpResponse(template.render(context))

The render() function takes the request object as its first argument, a template name as its second argument and a dictionary as its optional third argument. When you pass the context to the render method, the variables in the template get replaced by their dictionary values in the context.‘queue_list’ in the template gets replaced with the value of queue_list = Queue.objects.all(). queue_list isn’t actually a list but a ModelQuerySet which gives a future. The actually list is not obtained until get() is run. I can it also be invoked in the iterator, like the for loop in our HTML.The view then returns an HttpResponse object of the given template rendered with the given context.

Screen Shot 2015-02-12 at 7.13.51 PM

The information the user provides on this page is posted to the create_post view to handle.

def create_post(request):
    template = loader.get_template('tiktokadmin/create_post.html')
    text = request.POST['tweettext']
    queueid = request.POST['queue_id']
    tweetid = uuid.uuid4()
    if queueid != '0':
        tweet_time = request.POST['when_to_tweet']
        if tweet_time == '':
            template = loader.get_template('tiktokadmin/create_post_no_time.html')
            context = RequestContext(request)
            return HttpResponse(template.render(context))
        tweet_time = datetime.strptime(tweet_time, '%Y-%m-%d %H:%M:%S')
        Tweets_queue.create(queue_id = queueid, time_to_send = tweet_time, tweet_id = tweetid)
    Tweets.create(id = tweetid, created = datetime.utcnow(), modified = datetime.utcnow(),  tweet = text)
    context = RequestContext(request, {'text': text})
    return HttpResponse(template.render(context))

To come…

This is just the completion of Phase 1 of my TikTokTweet project, still have quite a bit of work to do. I haven’t done much of anything with the front end as you may be able to tell. I was considering trying out Bootstrap to make the app look sleek and shiny without too much hassle, given my limited experience with design in the front end. The other possibility would be JQuery. I also need to figure out a better way to do a datetime input as opposed to the yyyy-mm-dd hh:mm:ss string format I’m requesting from the user currently. I’m welcome any suggestions and, I’m pretty new to web application development and definitely eager to learn more. Up next, I’ll be looking into the backend parts of the app where the scheduler and lister reside. I think that’s where the real magic of this app lies.

Advertisements

About Rebecca Mills

Biochemist by trade, transitioning to computer engineering.
This entry was posted in Uncategorized. Bookmark the permalink.