Laravel 5.1 - Order by two columns not working as intended

I have two columns, quarter_number and quarter_year. The quarter_number column stores a value between 1 and 4, while quarter_year stores a year value. I want the data to be sorted like so for example:

ex: (quarter_number - quarter_year)
4 - 2015
3 - 2015
2 - 2015
1 - 2015
4 - 2014
3 - 2014
etc...

Thus it seemed like this statement would work:

$last_figures = QuarterData::where('company_id', '=', $company->id) ->orderBy('quarter_number')->orderBy('quarter_year')->get();

Unfortunately, it doesn't seem to work as intended (which I thought would be similar to a radix sort). It ends up just sorting by the year (or whatever to last orderBy statement is). Do I have to just program my own custom radix sort for this? Or is there a better way?

Thank you.