The annual Arab Youth Survey has been getting a lot of media attention since its findings were published earlier this month. Worldwide, it has generated a couple of hundred news items and three Emirati newspapers – Gulf News, The National and Khaleej Times – have produced no fewer than 17 items between them. In the US, the Washington Post alone has had three articles about it (here, here and here).

Much of this coverage has focused on the survey's not-very-surprising "discovery" that the vast majority of young Arabs interviewed disapprove of IS and have no intention of joining it. Some of the other findings are more interesting – for example, that a majority of youth in the GCC states (57%) support the nuclear deal with Iran. The introduction to the survey adds:

"While most young Arabs believe the Sunni-Shia divide has grown over the past five years, and is helping to fuel regional unrest, it seems the youth of today are increasingly concerned about the role they believe religion plays in fuelling conflict."

At first sight, the Arab Youth Survey is a serious piece of research but its credibility and independence are somewhat undermined by its links to the United Arab Emirates and by another of the survey's reported findings – that young Arabs view the UAE as "a model country that is economically secure, and is the most favoured nation to live in and set up a business". It is the fifth year running that the survey has come up with this result.
  

The survey was organised by ASDA’A Burson-Marsteller, a leading PR firm in Dubai which has previously won contracts from the Emirati government. Polling was done by Penn Schoen Berland, whose Middle East HQ is also in Dubai. Both firms belong to the international PR and advertising group, WPP.

The survey was endorsed in a series of tweets (here, here, here and here) by the ruler of Dubai, Sheikh Mohammed bin Rashid Al Maktoum:

• "Received a briefing today on the Arab Youth Survey conducted by international companies Asda'a Burson-Marsteller and Penn Schoen Berland."

• "For the fifth year in succession, in this survey of youth in 16 Arab countries, the UAE ranked first as a location to live and work."

• "The UAE was also ranked first by Arab youth as a model for successful development, followed by the US, Germany and France."

• "The UAE was also ranked by Arab youth as their preferred regional startup location. Our message to them is: You are most welcomed."

The last of these tweets prompted some subversive reactions from other Twitter users. One said: "Only if we 'Arab youth' could get a visa to the UAE." Another, pointing to a report from Amnesty International, commented: "There must be many Syrian youth who would love to resettle in the UAE." 

In the survey, interviewees were asked: "Which country in the world, if any, would you like to live in?" The reported answers were:

United Arab Emirates: 22%
United States: 15%
Germany: 11%
Saudi Arabia: 11%
Canada: 10%
France: 10%
United Kingdom: 9%
Turkey: 7%
Malaysia: 5%
Kuwait: 4%
Total: 104%

They were also asked: "Which country in the world, if any, would you most like your country to be like?" The reported answers were:

United Arab Emirates: 23%
United States: 19%
Germany: 12%
France: 10%
United Kingdom: 10%
Saudi Arabia: 9%
Canada: 9%
Turkey: 8%
Japan: 8%
Malaysia: 7%
Total: 115%

This leads the PR firm to claim, in its report of the survey, that the UAE "is the country most [of Arab youth] would like to live in" – which is untrue. The UAE did score better than other countries but less than a quarter said they would like to live there. Another way of interpreting it would be to say that more than three-quarters would NOT choose to live in the UAE.

It's also worth pointing out that the figures for both questions add up to more than 100%. The 104% total for the first question might be explained by rounding up results to the nearest whole number, but rounding up cannot explain the 115% total for the second question – so there is obviously something wrong with the figures.

One problem with the survey is that while ASDA’A Burson-Marsteller's account of its findings is strong on presentation, with lots of colourful infographics, there are issues with the underlying data and methodology that it doesn't address. This raises doubts as to whether the survey accurately reflects the views of "Arab youth" as a whole.

In its description of the methodology, ASDA’A Burson-Marsteller says the polling firm conducted 3,500 face-to-face interviews with Arabs aged 18 to 24, divided equally between male and female. It continues: "Respondents, exclusively nationals of each of the surveyed countries, were selected to provide an accurate reflection of each nation’s geographic and socio-economic make-up. The gender split of the survey is 50:50 male to female. The margin of error of the survey is +/-1.65 per cent."

So far, so good. But there are a couple of problems. One is that the interviews were conducted in large cities, though not just capital cities. ASDA’A Burson-Marsteller boasts about this, saying it "provides a more accurate national picture than findings based solely on the responses of those living in capital cities". That may be true, but it also means the survey only reflects the opinions of urban youth – excluding those in rural areas and smaller towns whose views are not necessarily the same.

A more serious problem, however, is the geographical spread of interviewees across the region. Sixteen countries were included in the survey (Syria was omitted because of the conflict there) and, country by country, the numbers of interviewees were as follows:
 

The effect of this is that Arabs in the GCC countries account for 40% of the sample – far more than can be justified on the basis of their population. The UAE, with a citizen population of less than two million, and Egypt, with a population estimated at 80-90 million, are both represented in the survey by the same number of respondents. 

Unless the national samples were weighted to take account of population differences, this would inevitably over-represent views from the least populous countries. The methodology described by ASDA’A Burson-Marsteller makes no mention of weighting for population, so we should probably assume there was none.

On that basis, it's difficult to regard the Arab Youth Survey as giving a balanced picture of views across the region.