Hey everyone, I wanted to give you a brief run down on a little project I’ve been planning. For a while I was doing this daily “Cassandra Pop Quiz” thing on my. Everyday at the same time, I would send out a pop quiz question of a Cassandra nature and people would respond. I was a lot of fun, but I times I would be busy or in meetings and I would forget, leading the question being sent out late. I got this idea that if I could automate my pop quiz question to be sent out on a schedule, as in the same time everyday. it would be easy and I could just forget about, knowing that it would be taken care of. This lead to the concept of “TikTokTweet”, a Twitter client for scheduling tweets. Tik Tok was the round mechanical man in the Land of Oz books. He was a true and faithful companion to dorothy, he would never let her down. He was always there when she was in need, like “clockwork” we’ll say. He would need to be wound up on a schedule to continue to his normal functioning. Hopefully this little “bot” will be equally as reliable.
Some the features I hope to include in TikTokTweet:
- Ability to create a Tweet
- you can save it in the database for it for later
- optionally add it directly to a queue to be tweeted out at a set time.
- Add your saved tweets to a queue
- assign a date and time to saved tweets to be sent out from the Schedule a Tweet page
- Choose a to particular queue to add the tweet to
- Create queues for tweets
- Create and name queues for a specific categories of tweet,
- add tweets to these queues to be sent out on schedule .
- add tweets directly from Create Tweet or add a saved tweet from the Schedule a Tweet page.
- Add new queues from the Manage queues page, and edit or delete queued tweets as well.
A brief overview of the application flow:
From the home page of the application, you have four options. If you are just starting out, you may want to go to the “Create new Tweet” part of the site, or potentially start out creating a new queue in “Manage Queues”. Lets have a look at creating a new tweet with the app as a starting point.
On the “Create new Tweet” page, we see are usual 140-char limit box that most of us are familiar with from Twitter. We can fill this out and send this into our database for later usage. Optionally, we can make use of that tweet right away and send it to a queue from our list(I’ll be explaining where we make those shortly) and attach a date and time for it to be tweeted out. Because I have yet to explain queuing, lets say we just submit this tweet to the database without any other options, and we’ll deal with it later.
Now say we want to add the tweet we sent to the database to a queue. We would click on the “Manage Tweets” page from the front page. Here we have the list of tweets sitting in our database. We will select a tweet and choose a queue from the list and date and time for it to be tweeted out.
Now lets looking at how queuing works. A logical step early in the game would be to create a queue to add tweets to. A queue is a list of tweets on a schedule to be tweeted out at different set time. We can select a queue to look at from the drop down list, or create a new one. Initially, there won’t be any queues until you create one.
Once we choose a particular queue to look at, we can make changes if we like. We can edit a tweet in the queue or remove it from the queue entirely (it will still exist in the database). We may even want to delete the entire queue.
If we choose to edit a tweet, we will be brought to the “Edit Tweet” page for that tweet. Here we have the option to change the text of that tweet, or even change it’s place in the queue, assigning it a new date and time to be tweeted out. Hit the “Submit” button for the changes to take effect.
Of course, we are going to want to have a look at the response to these tweets that we get. Responds are organized on a queue basis, and from the “Manage Response” page, we will select a queue for which we want to see the associated responses, as in responses for tweets in this queue.
From the next section of the “Manage Responses” pages, we will be able to see responses associated with the particular queue we have show, with the most recent response at the top. Because the timing of the responses is important to me, it’s necessary that I be able to get these responses in order. TikTokTweet should list responses in order from newest to oldest.
A little look at the potential data model:
Because this is Cassandra we are using, we need think about the kind of queries we be performing against the database before we can really start developing our application. Firstly I knew I would need a table as to store tweets in as I created them from the “Create Tweet” page. The plan was to be able to add these tweets to various queues which i had created, at and they would be tweeted out at a particular time. This would be set from the “Schedule a Tweet” page. Since a tweet can be in multiple queues, we’ll store those in their own table,
tweets. To keep track of and reference the list of queues I have created, I would need a
queues table, holding the list of queues. Once a queue is selected for a particular tweet, we set a date time for that tweet within the queue. I will need a
tweets_queue table to hold tweets and their associated dates/times for tweeting. I’ll use
‘CLUSTERING ORDER BY’ to store the dates in descending order, which will make the queries much easier. To select the next tweet to be sent, I’ll just have to do a
‘SELECT time_to_send, tweet_id FROM tweets_queue WHERE queue_id = <id> LIMIT 1’. Simple query now. A
tweets_sent table might also be necessary eventually to keep track of tweets that have already been sent to avoid redundancy. Finally, responses to these tweets would be held in
queue_tweet_responses table, which could be access on a per queue basis in descending time order, as can be seen from the manage responses pages.
/* Table to store tweets. There is timestamp for the initial * creation of the tweet and one for date/time modified incase any edits are made. */ CREATE TABLE tweets ( id UUID, tweet text, description text, created timestamp, modified timestamp, PRIMARY KEY(id) );
/* Table to store the different queues created. A queue is a list * of tweets of a particular category. Each queue type gets its * own unique id. */ CREATE TABLE queue ( id UUID, name text, created timestamp, modified timestamp, PRIMARY KEY (id) );
/* Table showing what tweets are associated with which queues and time for * the tweet to be sent out. Each tweet is in a particular queue with a * certain set time to be set out, hence the chosen primary key. * Clustering order for time_to_send is descending such that next tweet * to go out is at the top of the list. */ CREATE TABLE tweets_queue ( queue_id UUID, time_to_send timestamp, tweet_id UUID, PRIMARY KEY(queue_id, time_to_send) ) WITH CLUSTERING ORDER BY (time_to_send desc);
/* Tables to hold responses to tweets. TIMEUUID is used as a * distinct identifier here to deal with the unlikely possibility * that two people could reply to a tweet at the exact same time. * The queue_id is including in the primary key such that we can * select the responses for a particular queue. Ordering by time * descending is chosen such that the latest reply will be at the top of the list. */ CREATE TABLE queue_tweet_responses ( time_received TIMEUUID, tweet_id UUID, queue_id UUID, response text, PRIMARY KEY(queue_id, time_received) ) WITH CLUSTERING BY (time_received desc);
/* Table to hold tweets which have already been sent. This is so * we can keep track of tweets that have already been sent out. * Eventually this could be work to help us if the program crashes * and we don't end up sending the same tweets out again by accident. */ CREATE TABLE tweets_sent ( queue_id UUID, time_sent timestamp, tweet_id UUID, PRIMARY KEY(queue_id, time_sent) ) WITH CLUSTERING ORDER BY (time_sent desc);
I know I want TikTokTweet to be lightweight and easy to use, and enable me to manage questions and responses to my daily quiz questions so I could focus on other things. My plan is to write my app in Python, using Tweepy API for the backend features, and Django framework along with Django Cassandra Engine (a database wrapper for Django framework, using cqlengine).