Programming With Yii2: Automated Testing With Codeception

Final product image — What You'll Be Creating

If you're asking, "What's Yii?" check out my earlier tutorial, Introduction to the Yii Framework, which reviews the benefits of Yii and includes an overview of what's new in Yii 2.0, released in October 2014.

In this Programming With Yii2 series, I'm guiding readers in use of the Yii2 Framework for PHP. In this tutorial, I'll explore automated testing using Codeception, which is integrated with the Yii2 development framework.

Admittedly, my experience writing tests with my code is scarce. I've often been part of solo or small projects with limited resources. During my time at Microsoft, we had distinct test teams that did this. But frankly, this is likely typical of you too, right? Programmers like to code, they don't write tests—at least old school programmers didn't.

Codeception is an innovative library that literally aims to make writing tests fun and easy. And, I'd say they've succeeded to a reasonable degree. As I dipped my toe into the water of "Lake Codeception", it was mostly easy and fun. However, as I began to go deeper, I ran into configuration issues with Yii and the specific modules used in this series. There were definitely challenges. Overall, though, I'm impressed and see the benefit of learning more.

Put simply, Codeception and its integration with Yii make me want to write more tests, a first for me. I suspect you'll have a similar experience.

A little reminder before we get started, I do participate in the comment threads below. I'm especially interested if you have additional thoughts or want to suggest topics for future tutorials. If you have a question or topic suggestion, please post below. You can also reach me on Twitter @reifman directly.

Getting Started

Installing Codeception

To guide me, I used Yii's Testing Environment Setup documentation. I began with a global install of codeception so I could use it from any project.

$ composer global require "codeception/codeception=2.1.*"
Changed current directory to /Users/Jeff/.composer
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing symfony/yaml (v3.1.1)
    Loading from cache
...
codeception/codeception suggests installing symfony/phpunit-bridge (For phpunit-bridge support)
Writing lock file
Generating autoload files

You also need to require codeception/specify:

$ composer global require "codeception/specify=*"
Changed current directory to /Users/Jeff/.composer
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
composer require "codeception/verify=*"
  - Installing codeception/specify (0.4.3)
    Downloading: 100%         

Writing lock file
Generating autoload files

And codeception/verify:

$ composer require "codeception/verify=*"
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing codeception/verify (0.3.0)                  
    Downloading: 100%         

Writing lock file
Generating autoload files

Next, it helps to set up an alias for codecept using your global composer directory:

1	$ composer global status
2	Changed current directory to /Users/Jeff/.composer
3	No local changes

This sets the alias:

1	$ alias codecept="/Users/Jeff/.composer/vendor/bin/codecept"

Yii also requires you install Faker, which generates fake testing data for your application:

$ composer require --dev yiisoft/yii2-faker:*
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update                               
Generating autoload files

Setting Up Codeception With Your Application

Codecept bootstrap initializes codeception for your Yii application, creating a variety of configuration files for building and running tests against your application. We're using the Hello application from this series for this tutorial. See the GitHub link on this page to get the code.

$ codecept bootstrap
 Initializing Codeception in /Users/Jeff/Sites/hello 

File codeception.yml created       <- global configuration
tests/unit created                 <- unit tests
tests/unit.suite.yml written       <- unit tests suite configuration
tests/functional created           <- functional tests
tests/functional.suite.yml written <- functional tests suite configuration
tests/acceptance created           <- acceptance tests
tests/acceptance.suite.yml written <- acceptance tests suite configuration
tests/_output was added to .gitignore
 --- 
tests/_bootstrap.php written <- global bootstrap file
Building initial Tester classes
Building Actor classes for suites: acceptance, functional, unit
 -> AcceptanceTesterActions.php generated successfully. 0 methods added
\AcceptanceTester includes modules: PhpBrowser, \Helper\Acceptance
AcceptanceTester.php created.
 -> FunctionalTesterActions.php generated successfully. 0 methods added
\FunctionalTester includes modules: \Helper\Functional
FunctionalTester.php created.
 -> UnitTesterActions.php generated successfully. 0 methods added
\UnitTester includes modules: Asserts, \Helper\Unit
UnitTester.php created.

Bootstrap is done. Check out /Users/Jeff/Sites/hello/tests directory

For some reason, I also ended up with duplicate testing directories in hello/tests; just deleting hello/tests/functional, hello/tests/acceptance, and hello/tests/unit cleared things up. All the tests live in hello/tests/codeception/*.

The Different Kinds of Tests

Codeception is focused on three kinds of tests:

Unit testing verifies that specific units are working, such as an exhaustive test of all your model's methods.
Functional testing verifies common application scenarios as if a user was acting them out, but using web browser emulation.
Acceptance testing is identical to functional testing but actually runs the tests through a web browser.

And it supports three different kinds of test formats for your testing code:

Cept: it's the simplest single scenario test file
Cest: an object oriented format for running multiple tests within a single file
Test: tests written on PHPUnit, a PHP testing framework

Let's begin with an example of acceptance tests using cept format:

Acceptance Testing

We'll use Codeception's Welcome test example first.

1	$ codecept generate:cept acceptance Welcome
2	Test was created in /Users/Jeff/Sites/hello/tests/acceptance/WelcomeCept.php

This generates tests/acceptance/WelcomeCept.php, which we'll edit below.

Since acceptance tests require the browser, we have to edit /tests/acceptance.suite.yml in our project to provide our development URL, http://localhost:8888/hello:

# Codeception Test Suite Configuration
#
# Suite for acceptance tests.
# Perform tests in browser using the WebDriver or PhpBrowser.
# If you need both WebDriver and PHPBrowser tests - create a separate suite.

class_name: AcceptanceTester
modules:
    enabled:
        - PhpBrowser:
            url: http://localhost:8888/hello/
        - \Helper\Acceptance

Now, we're ready to modify the initial test in tests/acceptance/WelcomeCept.php. I'm writing a test that loads the front page to make sure that it works as expected.

Codeception tests have the concept of an actor, in this case, $I = new AcceptanceTester().

Here's how it describes actors in the Codeception documentation:

We have a UnitTester, who executes functions and tests the code. We also have a FunctionalTester, a qualified tester, who tests the application as a whole, with knowledge of its internals. And an AcceptanceTester, a user that works with our application through an interface that we provide.

You can comment your tests with code, such as $I->wantTo('perform a certain test') or 'ensure that the frontpage works'.

In my test, I want to see $I->see text for 'Congratulations!' and 'Yii-powered':

<?php
$I = new AcceptanceTester($scenario);
$I->wantTo('ensure that frontpage works');
$I->amOnPage('/');
$I->see('Congratulations!');
$I->see('Yii-powered');

Here's the current Hello home page:

Programming with Yii - The Hello Home Page

Next, let's run the test, simply codecept run:

1	$ codecept run
2	Codeception PHP Testing Framework v2.1.11
3	Powered by PHPUnit 5.3.5 by Sebastian Bergmann and contributors.
4	Acceptance Tests (1) ---------------------------------------------------------------------------------------
5	Ensure that frontpage works (WelcomeCept) Ok
6	------------------------------------------------------------------------------------------------------------
7	Functional Tests (0) ------------------------
8	---------------------------------------------
9	Unit Tests (0) ------------------------------
10	---------------------------------------------
11	Time: 554 ms, Memory: 10.25MB
12	OK (1 test, 2 assertions)

As you can see, our test passed, and the code to verify this functionality was quite readable and simple.

Notes on Yii's Default Tests

To go further, I began using Yii's default tests. At this point, I ran into a number of configuration issues—most due to my use of the custom yii2-user module in this series. Others were due to small bugs with Yii, which its team has been quick to respond to and fix when reported on GitHub; in some cases, issues had been fixed in later releases of the yii2-basic tree.

Also, because I'd updated the yii2-basic tree for this series, I had to make small changes to some of the default tests.

Here's an example of the output for running the default acceptance tests once I'd made some minor adjustments:

1	$ codecept run
2	Codeception PHP Testing Framework v2.1.11
3	Powered by PHPUnit 5.3.5 by Sebastian Bergmann and contributors.
4
5	Acceptance Tests (4) -------------------------------------------------------------------------------------------------
6	Ensure that about works (AboutCept) Ok
7	Ensure that contact works (ContactCept) Ok
8	Ensure that home page works (HomeCept) Ok
9	Ensure that login works (LoginCept) Ok
10	----------------------------------------------------------------------------------------------------------------------

Functional Testing

To get functional tests to work, I needed to run an instance of Yii's built-in server. I hadn't known about this component until Yii's Alex Makarov mentioned it in our GitHub exchange.

1	$ ./yii serve

I made small changes to the functional tests in /tests/codeception/functional, mostly to look for my specific updated text strings, i.e. "Invalid login or password" in place of Yii's default. Here's a look at LoginCept.php:

<?php
use tests\codeception\_pages\LoginPage;
$I = new FunctionalTester($scenario);

$I->wantTo('ensure that login works');

$loginPage = LoginPage::openBy($I);
$I->see('Login');
$I->amGoingTo('try to login with empty credentials');
$loginPage->login('', '');
$I->expectTo('see validations errors');
$I->see('Login cannot be blank.');
$I->see('Password cannot be blank.');

$I->amGoingTo('try to login with wrong credentials');
$loginPage->login('admin', 'wrong');
$I->expectTo('see validations errors');
$I->see('Invalid login or password');

$I->amGoingTo('try to login with correct credentials');
$loginPage->login('admin', 'admin11');
$I->expectTo('see user info');
$I->see('Logout');

Basically, the code accesses the LoginForm model and tests its various methods using Yii serve.

Here's the /tests/codeception_pages/LoginPage.php testing code it's leveraging (I also had to modify it for changes we've made to the series):

class LoginPage extends BasePage
{
    public $route = 'user/login';

    /**
     * @param string $username
     * @param string $password
     */
    public function login($username, $password)
    {
        $this->actor->fillField('input[name="login-form[login]"]', $username);
        $this->actor->fillField('input[name="login-form[password]"]', $password);
        $this->actor->click('button[type=submit]');
    }
}

You can see that we're coding the actor to fillFields and click buttons for our updated form fields.

While troubleshooting my Codeception integration with Yii, I found it helpful to run these tests in verbose mode:

1	$ codecept run -vvv

Here's the verbose output from the Login functional tests—in MacOS Terminal, PASSED and FAILED are color coded red or pink and inverted for visibility:

Functional Tests (4) -------------------------------------------------------------------------------------------------
Modules: Filesystem, Yii2
...
----------------------------------------------------------------------------------------------------------------------
Ensure that login works (LoginCept)
Scenario:
* I am on page "/index-test.php/user/login"

  [Page] /index-test.php/user/login
  [Response] 200
  [Request Cookies] []
  [Response Headers] {"content-type":["text/html; charset=UTF-8"]}
* I see "Login"
* I am going to try to login with empty credentials
* I fill field "input[name="login-form[login]"]",""
* I fill field "input[name="login-form[password]"]",""
* I click "button[type=submit]"
  [Uri] http://localhost/index-test.php/user/login
  [Method] POST
  [Parameters] {"_csrf":"VEpvcmk3bVgFH1Y9AVsmYWQQDEouTSggYXMFGStdKBEnCyQfBxo8Bw==","login-form[login]":"","login-form[password]":""}
  [Page] http://localhost/index-test.php/user/login
  [Response] 200
  [Request Cookies] {"_csrf":"dd395a9e5e3c08cfb1615dae5fc7b5ba0a2025c003e430ba0139b300f4a917ada:2:{i:0;s:5:"_csrf";i:1;s:32:"QU9OhlK90Zc8GzEx59jkBjEIsAKmn-Q_";}"}
  [Response Headers] {"content-type":["text/html; charset=UTF-8"]}
* I expect to see validations errors
* I see "Login cannot be blank."
* I see "Password cannot be blank."
* I am going to try to login with wrong credentials
* I fill field "input[name="login-form[login]"]","admin"
* I fill field "input[name="login-form[password]"]","wrong"
* I click "button[type=submit]"
  [Uri] http://localhost/index-test.php/user/login
  [Method] POST
  [Parameters] {"_csrf":"QjFBRl9hMjMTZHgJNw15CnJrIn4YG3dLdwgrLR0Ld3oxcAorMUxjbA==","login-form[login]":"admin","login-form[password]":"wrong"}
  [Page] http://localhost/index-test.php/user/login
  [Response] 200
  [Request Cookies] {"_csrf":"dd395a9e5e3c08cfb1615dae5fc7b5ba0a2025c003e430ba0139b300f4a917ada:2:{i:0;s:5:"_csrf";i:1;s:32:"QU9OhlK90Zc8GzEx59jkBjEIsAKmn-Q_";}"}
  [Response Headers] {"content-type":["text/html; charset=UTF-8"]}
* I expect to see validations errors
* I see "Invalid login or password"
* I am going to try to login with correct credentials
* I fill field "input[name="login-form[login]"]","admin"
* I fill field "input[name="login-form[password]"]","admin11"
* I click "button[type=submit]"
  [Uri] http://localhost/index-test.php/user/login
  [Method] POST
  [Parameters] {"_csrf":"bG8uMXdPYk49Ohd.HyMpd1w1TQkwNSc2WVZEWjUlJwcfLmVcGWIzEQ==","login-form[login]":"admin","login-form[password]":"admin11"}
  [Headers] {"location":["http://localhost/index-test.php"],"content-type":["text/html; charset=UTF-8"]}
  [Page] http://localhost/index-test.php/user/login
  [Response] 302
  [Request Cookies] {"_csrf":"dd395a9e5e3c08cfb1615dae5fc7b5ba0a2025c003e430ba0139b300f4a917ada:2:{i:0;s:5:"_csrf";i:1;s:32:"QU9OhlK90Zc8GzEx59jkBjEIsAKmn-Q_";}"}
  [Response Headers] {"location":["http://localhost/index-test.php"],"content-type":["text/html; charset=UTF-8"]}
  [Redirecting to] http://localhost/index-test.php
  [Page] http://localhost/index-test.php
  [Response] 200
  [Request Cookies] {"_csrf":"dd395a9e5e3c08cfb1615dae5fc7b5ba0a2025c003e430ba0139b300f4a917ada:2:{i:0;s:5:"_csrf";i:1;s:32:"QU9OhlK90Zc8GzEx59jkBjEIsAKmn-Q_";}"}
  [Response Headers] {"content-type":["text/html; charset=UTF-8"]}
* I expect to see user info
* I see "Logout"
 PASSED 

Overall, there's a bit to learn to get started with Codeception and properly code your tests. But the results are impressive and helpful.

Unit Testing

Basically, unit tests are programmatic testing of our infrastructure and models. Ideally, we would write tests for every method and usage variation of our models.

Unfortunately, I was not able to get unit tests to work within our tree because of either small Yii bugs yet to be released or configuration issues between Codeception and yii2-user which we integrated in How to Program With Yii2: Integrating User Registration.

1	Unit Tests (3) ----------------------------------------------------------------------------------------------------
2	Modules:
3	-------------------------------------------------------------------------------------------------------------------
4	Trying to test login no user (tests\codeception\unit\models\LoginFormTest::testLoginNoUser)...
5	<pre>PHP Fatal Error 'yii\base\ErrorException' with message 'Call to undefined function
6	tests\codeception\unit\models\expect()'

I'll address unit testing again in our Startup series which does not use yii2-user but instead uses the Yii Advanced tree's built-in user integration.

Let's look at a couple of examples from the Yii2-app-basic tree.

Testing Contact Form Emails

The hello/tests/codeception/unit/models/ContactFormTest.php tests sending an email through programmatic use of models:

<?php

namespace tests\codeception\unit\models;

use Yii;
use yii\codeception\TestCase;
use Codeception\Specify;

class ContactFormTest extends TestCase
{
    use Specify;

    protected function setUp()
    {
        parent::setUp();
        Yii::$app->mailer->fileTransportCallback = function ($mailer, $message) {
            return 'testing_message.eml';
        };
    }

    protected function tearDown()
    {
        unlink($this->getMessageFile());
        parent::tearDown();
    }

    public function testContact()
    {
        $model = $this->createMock('app\models\ContactForm', ['validate']);
        $model->expects($this->once())->method('validate')->will($this->returnValue(true));

        $model->attributes = [
            'name' => 'Tester',
            'email' => 'tester@example.com',
            'subject' => 'very important letter subject',
            'body' => 'body of current message',
        ];

        $model->contact('admin@example.com');
        
        $this->specify('email should be send', function () {
            expect('email file should exist', file_exists($this->getMessageFile()))->true();
        });

        $this->specify('message should contain correct data', function () use ($model) {
            $emailMessage = file_get_contents($this->getMessageFile());

            expect('email should contain user name', $emailMessage)->contains($model->name);
            expect('email should contain sender email', $emailMessage)->contains($model->email);
            expect('email should contain subject', $emailMessage)->contains($model->subject);
            expect('email should contain body', $emailMessage)->contains($model->body);
        });
    }

    private function getMessageFile()
    {
        return Yii::getAlias(Yii::$app->mailer->fileTransportPath) . '/testing_message.eml';
    }

}

I was unable to successfully get this test to pass because of a small bug in Yii which hasn't been updated yet (or at least I couldn't find the updated code). My drop of the Yii codebase was naming outbound email with date stamps and the code above was looking for a fixed filename. Thus, it always failed. Still, it's useful to see how programmatic testing can use models to generate a file and then look for that file and validate its contents to verify that code is working.

Testing Login

Let's look at hello/tests/codeception/unit/models/LoginFormTest.php. Again, my use of yii2-user made it overly difficult to integrate at the time of writing this tutorial; however, we can look at the conceptual approach to unit testing user model functions.

Here's testLoginCorrect(), which looks to see if login succeeds with a correct password:

public function testLoginCorrect()
    {
        $model = new LoginForm([
            'username' => 'admin',
            'password' => 'admin11',
        ]);

        $this->specify('user should be able to login with correct credentials', function () use ($model) {
            expect('model should login user', $model->login())->true();
            expect('error message should not be set', $model->errors)->hasntKey('password');
            expect('user should be logged in', Yii::$app->user->isGuest)->false();
        });
    }

It uses the LoginForm model to programmatically log in the user, and then it programmatically looks to see if Yii's current user is now no longer a guest.

1	expect('user should be logged in', Yii::$app->user->isGuest)->false();

What's Next?

I hope that you've enjoyed learning about Codeception and its integration with Yii, despite some of the roadblocks I ran into. Default installation of yii2-basic today should perform better.

If you'd like to read more about deciding when and what to test and why, I recommend reading Yii's Testing Overview. There's certainly more to learn about Codeception and writing more complete tests.

Watch for upcoming tutorials in our Programming With Yii2 series as we continue diving into different aspects of the framework. If you'd like to know when the next Yii2 tutorial arrives, follow me @reifman on Twitter or check my instructor page.

You may also want to check out our Building Your Startup With PHP series, which is using Yii2's advanced template as we build a real-world application. In fact, you can try out the startup application, Meeting Planner, today.