Using the Twitter API to Tweet Repetitive Content

Final product image — What You'll Be Creating

Welcome back to our coverage of the Twitter API. If you use Twitter, you may have come across a friend sharing tweets from the @infinite_scream bot (shown above). I know it's mostly a bot because it tweets at ten-minute intervals. But it varies the string length of its virtual screams to avoid being blocked by the Twitter's API's infamous undocumented restrictions. Tweet too frequently or repeat the same content and you'll find your bots hopelessly blocked.

Recently, an online friend asked me to help them write code for a bot that might repeat but provide some intelligent content variation. In today's tutorial, I'll write about how to do this with the Twitter API.

In today's tutorial, I'll describe how to build a bot that does the basics:

random tweet interval
random selection of status text
random selection of hashtags
random inclusion of URLs
avoid getting blocked by Twitter

However, if you want to use a Twitter API bot to effectively promote your product or service on your own account without annoying your followers, you need to write code that intelligently varies the topic, contents and frequency of your tweets in an organized way. I'll be writing about how to do that in future episodes.

If you have any questions or ideas, please post them in the comments below. If you'd like to see some of my other Envato Tuts+ tutorials, please visit my instructor page, especially my startup series. Let's get started.

Getting Started

For the bot in episode one, I'm trying to generate fun tweets on a regular basis while avoiding upsetting the great Twitter God of Restricted Access in the Sky.

The tweet content is very simple and can be randomly created by combining previously written status text, hashtags, and URLs.

The bot runs in Yii, a popular PHP-based platform. I'll keep the guide below fairly simple for straight PHP developers. However, I encourage you to use frameworks. You can learn more in my Yii Series.

Building the Bot

Registering a Twitter App

Basically, the first thing I did was register an app to get my Twitter keys:

Repeating Twitter API - Setting up your Twitter App

If you aren't familiar with creating an app and authorizing API access with Twitter, please review some of our earlier tutorials:

Building With the Twitter API: Getting Started
Learn the Twitter API with PHP using a standalone, open-source Yii application called Birdcage.

Jeff Reifman
03 Nov 2014

Twitter API
Building With the Twitter API: Tweet Storms
Some ideas are too big for 140 characters. Post your thoughts in numbered sequences of tweets with the Twitter API. In this tutorial, we'll work through the...

Jeff Reifman
07 Jan 2015

Twitter API
Building Your Startup With PHP: Simplifying Onramp With OAuth
To make it easier for new users, we'll allow them to sign up with their Google, Facebook and Twitter accounts and let existing users connect these services...

Jeff Reifman
19 Sep 2016

Yii

Authoring Content for a Range of Tweets

I wanted to create a system where my friend (or any approved author) could write variations of tweets and place them in a database for ongoing use. First, I created a database migration to build the table for them.

All of my tables for this project have the prefix norm_. Here is the Tweet table or norm_tweet:

<?php
  use yii\db\Schema;
  use yii\db\Migration;
  class m170120_211944_create_table_norm_tweet extends Migration
  {
    public function up()
    {
        $tableOptions = null;
        if ($this->db->driverName === 'mysql') {
            $tableOptions = 'CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE=InnoDB';
        }

        $this->createTable('{{%norm_tweet}}', [
            'id' => Schema::TYPE_PK,
            'tweet' => Schema::TYPE_TEXT.' NOT NULL',
            'media_id' => Schema::TYPE_STRING.' NOT NULL',
        ], $tableOptions);

    }

    public function down()
    {

      $this->dropTable('{{%norm_tweet}}');
    }
  }

Next, I used Yii's Gii scaffolding system to create a model and CRUD files for me. Here's the model:

Repeating Twitter API - Yiis Gii Scaffolding Model Generator

And here's the CRUD generation:

Repeating Twitter API - Yiis Gii Scaffolding CRUD Generator

So all of this code automatically provides the grid view below and editing capabilities.

Repeating Twitter API - The Authoring Tweets UX

For now, I'm not using the Media ID, which is for images that are uploaded to be used within tweets. I'll likely address this in the next episode.

Pretty straightforward so far, right?

Adding a Range of Hashtags

Then, I repeat this process for the other models. Here's the norm_hash migration for hashtags:

<?php
  use yii\db\Schema;
  use yii\db\Migration;
  class m170120_212009_create_table_norm_hash extends Migration
  {
    public function up()
    {
        $tableOptions = null;
        if ($this->db->driverName === 'mysql') {
            $tableOptions = 'CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE=InnoDB';
        }

        $this->createTable('{{%norm_hash}}', [
            'id' => Schema::TYPE_PK,
            'tag' => Schema::TYPE_STRING.' NOT NULL',            
        ], $tableOptions);

    }

    public function down()
    {

      $this->dropTable('{{%norm_hash}}');
    }
  }

The idea is to randomly include a selected hashtag (or two) in the tweets to make it appear that the bot is human, varying its tweets.

Here's the Hashtag UX:

Repeating Twitter API - The Authoring Hashtags UX

I won't repeat the Yii Gii steps from above, but I repeat them for norm_hash and norm_url as well.

Adding a Variety of URLs

Here's the database migration for adding URLs:

<?php
  use yii\db\Schema;
  use yii\db\Migration;
  class m170120_211955_create_table_norm_url extends Migration
  {
    public function up()
    {
        $tableOptions = null;
        if ($this->db->driverName === 'mysql') {
            $tableOptions = 'CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE=InnoDB';
        }

        $this->createTable('{{%norm_url}}', [
            'id' => Schema::TYPE_PK,
            'url' => Schema::TYPE_STRING.' NOT NULL',
            'last_used' => Schema::TYPE_INTEGER . ' NOT NULL',
            'created_at' => Schema::TYPE_INTEGER . ' NOT NULL',
            'updated_at' => Schema::TYPE_INTEGER . ' NOT NULL',
        ], $tableOptions);

    }

    public function down()
    {

      $this->dropTable('{{%norm_url}}');
    }
  }

The bot master may add URLs at different times. It's probably best for this bot not to use old URLs or to repeat them. The last_used and created_at Unix timestamps allow the tweeting algorithm described below to decide when to use URLs.

Here's the URL UX page:

Repeating Twitter API - The Adding URLs UX

Now, let's look at the fun algorithm to combine all of these tables into interesting tweets that aren't annoying to Twitter's Master Overlord of Restrictions.

Randomizing the Content of Bot Tweets

It's fun to have a bot that tweets a variety of content, but the variety is also helpful at preventing it from getting blocked by Twitter.

You can see the Twitter Rate Limits here, but some of the rules for content repetition don't appear to be documented:

Repeating Twitter API - Rate Limits Chart

I took directions from my friend as to how they wanted the algorithm to build tweets from the database of tweets, hashtags, and URLs.

Here's the build algorithm we decided on for now; it's easy to tweak. I'll describe it in parts.

In my opinion the algorithm below is low on its use of hashtags and URLs, and if you want a more robust variety of content, change it to your liking.

First, we use yii\db\Expression; to properly select a random single tweet text from the table:

<?php

namespace frontend\models;

use Yii;
use yii\db\Expression;

...

public static function build() {
  // pick a random tweet message (any one of them)
  $txt = NormTweet::find()
  ->orderBy(new Expression('rand()'))
  ->one();

Then we decide whether to use a hashtag (currently 1 in 5 or 20% of the time) and how many to use (currently fixed to just one):

// decide whether to use a hashtag
// pick a random # between 0 and 4
$useHash = rand(0,4);
// if rand# is not 0 but instead 1,2,3 or 4, include hashtag(s)
// for less freq use of hash tags, we can change this > 2 or >3
if ($useHash>3) {
  // so we're now going to decide which and how many hash tags to
  // Creator decided to only use one hashtag for right now
    $numHash = 1; // rand(1,3);
    // select this rand# $numHash randomly from our list
    $hash = NormHash::find()
    ->orderBy(new Expression('rand()'))
    ->one();
} else {
  // don't use any hashtags
  $hash=false;
}

Then, we decide if there is a URL available to use. URLs must be less than a week old and they can only be used once every 72 hours (3 days). So any new URL might only be available for use once, twice or possibly three times before expiring.

// only use a url if it's less than a week old
  $url_weekago = time()-7*24*3600;
  $url_3daysago = time()-3*24*3600;
  // only use a url if it's not been posted in the last 72 hrs
  $url= NormUrl::find()
    ->where('created_at>'.$url_weekago)
    ->andWhere('last_used<'.$url_3daysago) // handles unused zero case
    ->orderBy(['id' => SORT_DESC])->one();

Finally, we build the tweet based on the selected data (available URLs are added only one in four times or 25% chance):

$content = $txt->tweet;
  if ($hash!==false) {
    $content.=' #'.$hash->tag;
  }
  // only add URL 1/4 of the time
  if (!is_null($url) && rand(1,4) ==1) {
    $content.=' '.$url->url;
    $url->last_used = time();
    $url->update();
  }
  return $content;

Choosing When to Tweet

Yii allows you to call console controllers from cron. So I add a call to my /console/DaemonController.php in crontab.

$ sudo crontab -l

Here's how my tasks are scheduled:

# m h  dom mon dow   command
*/3 * * * * /var/www/bot/yii daemon/frequent
*/15 * * * * /var/www/bot/yii daemon/quarter
0 * * * * /var/www/bot/yii daemon/hourly
15 1 * * * /var/www/bot/yii daemon/overnight
15 3 * * 5 /var/www/bot/yii daemon/weekly

Every hour, daemon/hourly in /console/DaemonController.php is requested. Our bot app only decides whether to tweet or not once every four hours.

First, you'll see I have a NormLog table which I didn't describe above, but that tracks all the output and when tweets were made. So my friend didn't want to tweet more than once a day.

public function actionHourly() {
  // every hour
  $current_hour = date('G');
  if ($current_hour%4) {
    // every four hours
    echo "Review tweeting plan...";
    $dayAgo = time()-24*3600;
    $nl= \frontend\models\NormLog::find()
      ->orderBy(['id' => SORT_DESC])
      ->one();
    echo 'created_at: '.$nl->id.' '.$nl->created_at.'...';
    echo 'dayago: '.$dayAgo.'...';
    if ($nl->created_at<$dayAgo) {
      // okay to maybe tweet
    ...

We didn't want followers of our bot to get annoyed by high frequency tweeting.

Then, we pick a number, basically the six times a day (every four hours), and tweet if the number is 6 (or a one in 12 chance).

  // every four hours, if 6 is picked from 1-12, we tweet
      // 1 in 12 chance 12x in two days
      $r = rand(1,12);
      if ($r==6) {
        \frontend\models\NormTweet::deliver();
        echo 'tweet, 6 was picked...';
      } else {
        echo "do not tweet, 1 in 12 # not picked...";
      }
    } else {
      // never tweet twice in 24 hrs
      echo 'do not tweet, not yet 24 hrs...';
    }
    echo '..made it to end...!';
  }
      if ($current_hour%6) {
      // every six hours
    }
    }

Delivering the Tweet to the Twitter API

Here's the NormTweet::deliver() method called by the Daemon to post the tweet:

public static function deliver()
{
    // post an update
    // construct
    $content = NormTweet::build();
    // tweet it using params for norm acct
    $connection = new TwitterOAuth(
        Yii::$app->params['norm']['consumer_key'],
        Yii::$app->params['norm']['consumer_secret'],
        Yii::$app->params['norm']['access_token'],
        Yii::$app->params['norm']['access_token_secret']);
    $postit = $connection->post("statuses/update",
        ["status" => $content]);        
    // save it in the log
    $nl = new NormLog();
    $nl->tweet = $content;
    $nl->save();
  }
}

The account's Twitter application keys are stored in /bot/frontend/config/params-local.php, configured from the bot.ini file I use:

$ more params-local.php 
<?php
return [
'norm' => [
     'consumer_key' => $config['norm_consumer_key'],
     'consumer_secret' => $config['norm_consumer_secret'],
     'access_token' => $config['norm_access_token'],
     'access_token_secret' => $config['norm_access_token_secret'],
     'user_id' => $config['norm_user_id'],
   ],

];

Bots aren't simple, but they are fun!

Examining the Results

Here are the results of our bot:

Repeating Twitter API - Tom McFarlin Head Editorial Goddess

Just kidding! That's one of the editorial goddesses, Tom McFarlin. AI scripts aren't yet capable of replacing his "insights," but Envato Tuts+ has hired me to work on this.

Here's the actual bot, meant to remind my friend and its followers that America's new politics aren't exactly normal. I imagine whatever your views you'd agree with that.

Repeating Twitter API - Bot Tweet Stream from Tutorial

I hope you've enjoyed this episode.

What's Next?

Next, I'm going to create a more marketing-driven platform to help you use the Twitter API to promote your startup, services, and business without getting labeled as a bot and blocked.

If you have any questions or suggestions about this tutorial, please post them in the comments. If you'd like to keep up on my future Envato Tuts+ tutorials and other series, please visit my instructor page or follow @reifman.