Mascote

Notes and tips to remember.

Crazy Fast Deployment of Composer Based Applications With Capistrano 3

If you already use capistrano you already know about the linked_dirs and probably use something like this to keep your PHP libraries across releases:

1
set :linked_dirs, fetch(:linked_dirs, []).push('vendor')

But sometimes you need to run composer to keep add another dependency and sometimes takes a lot of time to download everything or some composer server stop to respond or suffer a slowdown and your deploy breaks.

I developed a deployment strategy to not forget to run composer or install all dependencies everytime.

The ideia is have the composer.phar available before the deploy starts and faster deploy times. (< 30 seconds)

The magic start with the rake file task who is only invoked one time and only if necessary, so we can install composer just once:

1
2
3
file "/tmp/composer.phar" do |f|
  sh "wget -q https://getcomposer.org/composer.phar -O #{f.name}"
end

Here is another trick, if you do not put the composer.lock file on your repository, composer will try to install all the dependencies and create the lock file, so here whe create the file and let composer update their content later to speed next deployments:

1
2
3
file "/tmp/composer.lock" do |f|
  touch f.name
end

The remote_file set the first argument as a prerequisite and if the file no exist the file task with the name '/tmp/composer.phar' will be called. The '/tmp/composer.phar' is a rake task name and also a path. You can see more about remote_file here.

The file task will run and the file will be created on the shared_path, otherwise, nothing happens.

1
2
remote_file 'composer.phar' => '/tmp/composer.phar', roles: :app
remote_file 'composer.lock' => '/tmp/composer.lock', roles: :app

And we link these tasks above on the check flow who happens before the deploy to ensure the capistrano.phar and capistrano.lock will be available on the shared_path and linked to the release_path later:

1
2
3
4
5
namespace :deploy do
  namespace :check do
    task :linked_files => ["composer.phar", "composer.lock"]
  end
end

Finally we always run composer to ensure the dependencies will be on available to the application:

1
2
3
4
5
6
7
8
9
10
11
namespace :deploy do
  #...
  desc "updates composer"
  after :updated, :vendor do
    on roles(:app), in: :groups, limit: 3 do
      within release_path do
        execute :php, "composer.phar install --no-dev --optimize-autoloader"
      end
    end
  end
end

TL;DR

The whole thing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#...

set :linked_files, fetch(:linked_files, []).push('composer.phar', 'composer.lock')
set :linked_dirs, fetch(:linked_dirs, []).push('vendor')

file "/tmp/composer.phar" do |f|
  sh "wget -q https://getcomposer.org/composer.phar -O #{f.name}"
end

file "/tmp/composer.lock" do |f|
  touch f.name
end

remote_file 'composer.phar' => '/tmp/composer.phar', roles: :app
remote_file 'composer.lock' => '/tmp/composer.lock', roles: :app

namespace :deploy do
  namespace :check do
    task :linked_files => ["composer.phar", "composer.lock"]
  end

  desc "updates composer"
  after :updated, :vendor do
    on roles(:app), in: :groups, limit: 3 do
      within release_path do
        execute :php, "composer.phar install --no-dev --optimize-autoloader"
      end
    end
  end
end

Try to remove one or another dependency from the vendor directory and see composer installing only the missing piece on the next deploy!

You can find me on Twitter as @fgbreel.

See you next time!

Solving ‘Undefined Method Authenticate’ When Using MiniTest and Devise in Rails 4.2

If you are using methods like user_signed_in? inside your controllers and receiving errors like those when running MiniTest:

1
ActionView::Template::Error: undefined method `authenticate' for nil:NilClass

or

1
ArgumentError: uncaught throw :warden

just reopen ActionController::TestCase and include the Devise::TestHelpers putting this at the end of your test_helper.rb file:

1
2
3
class ActionController::TestCase
  include Devise::TestHelpers
end

No more errors!

tip: you can use the sign_in helper provided by Devise to use inside your controller tests.

HTTP and HTTPS Scheme Redirect With HAProxy.

If you are deploying HAProxy in front of some other web server like Varnish or Nginx and need to ensure certain URI will be accessed only over a secure connection can easily done with HAProxy.

Create a file with the URIs who need to be secured, one per line:

1
2
3
4
5
vim /etc/haproxy/secure-locations.map

/admin
/secure-area
/foo

and put on your frontend definition on haproxy.cfg:

1
2
3
frontend www-http
  ...
  redirect scheme https if { capture.req.uri,map_beg(/etc/haproxy/secure-locations.map) -m found }

and this into your secure frontend definition if you don’t want another places to follow the current (https) scheme:

1
2
3
frontend www-https 
  ...
  redirect scheme http if !{ capture.req.uri,map_beg(/etc/haproxy/secure-locations.map) -m found }

Cheers!

Task to Show Deployed Commits, Revisions and Branch With Capistrano 3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
desc "Show the difference between deployed revisions on server."
task :diff_report do
  on roles(:app) do
    current, previous, branch = fetch(:current_revision), fetch(:previous_revision), fetch(:branch)
    puts "\n" << "-"*63
    puts "===== [ \033[1;36m#{fetch(:application).capitalize} - #{fetch(:stage).capitalize}\033[0m ]"
    puts "=== Deployed Branch: \033[1;32m#{branch}\033[0m"
    puts "=== Deployed Revision: \033[1;32m#{current}\033[0m"
    puts "=== Previous Revision: \033[1;32m#{previous}\033[0m\n\n"

    # If deployed master branch, show the difference between the last 2 deployments
    # or show the difference between master and the deployed branch.
    base_rev, new_rev = branch != "master" ? ["master", branch] : [previous, current]

    # Show difference between master and deployed revisions.
    if (diff = `git log #{base_rev}..#{new_rev} --oneline`) != ""
      # Colorize refs
      diff.gsub!(/^([a-f0-9]+) /, "\033[1;32m\\1\033[0m - ")
      diff = "    " << diff.gsub("\n", "\n    ") << "\n"
      # Indent commit messages nicely, max 80 chars per line, line has to end with space.
      diff = diff.split("\n").map{|l|l.scan(/.{1,120}/).join("\n"<<" "*14).gsub(/([^ ]*)\n {14}/m,"\n"<<" "*14<<"\\1")}.join("\n")
      puts "=== Difference between current revision and deployed revision:\n\n"
      puts diff
    else
      puts "=== Deployed the last revision again.\n\n"
    end
  end
end

after :log_revision, :diff_report

Kudos for @ndbroadbent for the original post.

Configuring Automatic Package Upgrade on Debian/Ubuntu

UPDATE: To enable without pain just run: dpkg-reconfigure unattended-upgrades.

In order to enable automatic package upgrades you need to install unattended-upgrades package from repository:

1
2
3
aptitude install unattended-upgrades
#or
apt-get install unattended-upgrades

Then configure what will be upgraded periodicaly editing the file /etc/apt/apt.conf.d/50unattended-upgrades. The file is well commented, so in this case I configured to install security upgrades only and an email address to receive alert in case of problems:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Automatically upgrade packages from these (origin:archive) pairs
Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security"; 
//    "${distro_id}:${distro_codename}-updates";
//    "${distro_id}:${distro_codename}-proposed";
//    "${distro_id}:${distro_codename}-backports";
};

...

// Send email to this address for problems or packages upgrades
// If empty or unset then no email is sent, make sure that you
// have a working mail setup on your system. A package that provides
// 'mailx' must be installed.
Unattended-Upgrade::Mail "[email protected]";

Later configure the file /etc/apt/apt.conf.d/10periodic:

1
2
3
4
5
6
7
8
// Enable periodic
APT::Periodic::Enable "1";
// Download packages every n-days.
APT::Periodic::Download-Upgradeable-Packages "1";
// Refresh package list automaticaly every n-days, in this case, everyday.
APT::Periodic::Update-Package-Lists "1";
// Enable Unattended Upgrades to run every n-days, well, you know :)
APT::Periodic::Unattended-Upgrade "1";

That is.

How to Change Timezone on Ubuntu/Debian

To select another timezone for your Debian/Ubuntu based linux box run the following command and select your Country/Province.

1
dpkg-reconfigure tzdata

Just that!

Deploy With Capistrano Without SCM

To deploy with Capistrano withou using a SCM, you need to add these lines on your deploy.rb:

1
2
3
set :scm, :none             # If you set this line without the lines below, you will get an error with 'which' command when run cap deploy:check 
set :repository,  "."       # set the directory who will be compressed to deploy
set :deploy_via, :copy      # specify to use via compressed tar 

Solving Problem With Symlinks and Mac OS X on Samba

Some Mac OS X machines has a problem with symlinks on Samba, to fix this just add to your /etc/samba/smb.conf:

1
2
unix extensions = no
wide links = yes

And reload the smbd daemon.

Cheers.

New Life for a Dead Kindle Fire HD

After a failed attempt to write a bootloader using a wrong image in Kindle Fire HD 7“ from a friend, was totally dead, black screen, no sign of life.

As I went through this with my Galaxy Nexus who also has an OMAP4 CPU, I decided to follow the same steps to retrieve it using the usbboot. For this it was necessary to remove the battery and solder a small wire connecting pin USB Boot to GND to be recognized as omap44xx usb device in my linux PC.

The pin in question is below:

1
2
3
_______________* * 
this pin ---->[*]*
_______________* * 

After that, just download and compile the usbboot through the sources that can be found here.

Good luck.

Replacing a Failing Disk on CentOS 6

Create a identical partition map from the original disk:

1
# sfdisk -d /dev/sda | sfdisk --force /dev/sdb

search the UUID of /boot partition:

1
2
3
4
# dumpe2fs -h /dev/sda1 | grep UUID

dumpe2fs 1.42 (29-Nov-2011)
Filesystem UUID:          d22792ce-822a-4eeb-9e4f-1522214026c6

create a same filesystem type in new disk:

1
# mkfs.ext4 -U d22792ce-822a-4eeb-9e4f-1522214026c6 /dev/sdb1

copy /boot content to new disk mounted at /mnt:

1
2
3
# mount /dev/sdb1 /mnt

# rsync -avrP /boot /mnt

set the new disk as hd0 on grub:

1
2
3
# grub 
grub> find /grub/stage1
grub> device (hd0) /dev/sdb

We made the second drive /dev/sdb device (hd0) because putting grub on it this way puts a bootable mbr on the 2nd drive and when the first drive is missing the second drive will boot.

install grub on new device:

1
2
3
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

add the new disk partition to volume group:

1
# vgextend VolGroup00 /dev/sdb2

move content from old to new disk:

1
# pvmove /dev/sda2 /dev/sdb2

remove old disk partition from volume group:

1
# vgreduce VolGroup00 /dev/sda2

now, you can remove the old disk without reboot, to disable the disk to remove safely:

1
# echo 1 > /sys/block/sda/device/delete

enjoy!