Kevin is the future of the far right in Sweden. At 16, the computing student – who sports a beanie hat and a wispy beard – is active in the youth wing of the Sweden Democrats, the anti-immigration party that has forced a snap election after flexing its muscles as the country's third party. "There are not enough jobs for Swedish people, but there are more and more immigrants, hundreds a year coming into this town," he says in a shopping centre in Kristianstad, southern Sweden, an electoral stronghold of the far right and home to the party's national headquarters. "We are not a racist or fascist party," Kevin says. "There were racists and fascists when Jimmie was a boy, but they have all gone."

Jimmie is the Sweden Democrats' leader, Jimmie Åkesson, who lives in the area. Åkesson claims to have kicked the extremists out of the party, pulled up its roots in white-supremacist and neo-Nazi activism, and turned it into a slick electoral machine that has doubled its vote every four years, taking 13% in September's election. This self-image of the Sweden Democrats was challenged last weekend by the prime minister, Stefan Löfven, who called the party "neo-fascist", citing its early links with Keep Sweden Swedish and White Aryan Resistance.

Löfven, leader of the once mighty Social Democrats, now heads a lame-duck minority coalition with the Greens. He lashed out at his far-right nemesis as the political establishment scrambles to find a way of slowing the Sweden Democrat insurgency. The most recent opinion poll by YouGov suggests the far right is riding high and could take as much as 18% of the vote in new elections in March. Löfven's "neo-fascist" attack sparked a storm of debate, with critics saying that labelling the far right is irrelevant or dangerous. But the prime minister also expressed a strength of feeling against the Sweden Democrats that is shared by many. When it emerged on Friday that the party wants to lift a ban on recording the national origins of people in the criminal justice system, the head of Sweden's Bar Association said on Twitter they might as well put a badge on offenders' jackets – in reference to the practice in Germany in the 1930s. In Kristianstad, however, voters say they have heard it all before.

Lars-Åke, 55, a cleaner, voted for the Sweden Democrats because he approves of their policy of slashing immigration by 90%. "There are too many refugees — so many Arabs it feels like I need to learn Arabic," he says. He admits to a concern that the party is still "too extreme"as a result of its shady past: "They used to have some very bad people and a lot of them quit, but maybe a few are still left."

Denis, 18, is weighing up whether to vote for the centre left or for the Sweden Democrats. "They are neither racist nor fascist; they just talk about something no one else talks about – we have a problem with immigrants," the student said.

Politicians and the media in Sweden have maintained a stance unique in Europe in shunning the Sweden Democrats, excluding them from political debate and directing fierce criticism at anyone who suggests that the country's asylum policy is a problem. Sweden currently accepts more than twice as many refugees per capita as any other of the 34 member states of the OECD.

Fredrik Reinfeldt, the centre-right former prime minister who has called on Swedes at the election to "open your hearts" to refugees, made headlines again when he said the country had "more space than you can imagine" to receive asylum seekers, who are expected to number 80,000 this year – a new Kristianstad every year. But as cracks have begun to appear in the Nordic model that has served Sweden so well, the country's pro-immigration consensus has come under pressure. In Kristianstad itself, where 12,000 residents were born abroad, one in five voters chose the Sweden Democrats in September, and almost everyone has friends or workmates who are sympathisers. Even the party's opponents hesitate to call it fascist. "This is not a neo-fascist party," said Dr Anders Sannerstedt, a political scientist at Lund University. "As a party they have cut their links to their unpleasant roots. There might still be unpleasant individuals, but the party programme has no traces of the white supremacy movement."

Surveys appear to show that the number of Swedes with strongly anti-immigration values is coming down gradually, says Markus Uvell, head of the Kreab communications firm in Sweden. "What has happened instead is that anti-immigrant opinion has become mobilised, and people who put their anti-immigrant feelings second to other concerns are now allowing it to determine how they vote," Uvell says.

However, academic research this month suggests that Sweden Democrat voters hold "drastically more negative views" about Muslims than other voter groups and are motivated "purely by anti-minority sentiments" rather than the anti-establishment populism in which it is cloaked.

"The risk is that in a society where the Sweden Democrats have a greater and greater say and immigration is portrayed as basically a problem, racist ideas and actions become more normalised," says Daniel Poohl, editor of the anti-racist magazine Expo, who says racism and intolerance are a basic part of the party's identity. The local paper in Kristianstad recently had to suspend its practice of allowing online comments under articles because there was so much race hate.

Oscar Sjöstedt, the Sweden Democrats' economic spokesman, said in an email: "I think it is the most absurd statement a Swedish PM has made in a very, very long time." The party had made it "abundantly clear" that its core values included a belief in democracy, human rights and the rule of law, "values which, I presume, a 'neo-fascist' would frown upon".

It is not just their roots that put the Sweden Democrats at the radical end of the spectrum of far-right Nordic populism – echoes of its past can be heard in its political statements now. One party ideologue wrote about "the great decisive battle for our civilisation, our culture and our nation's survival". A theme of Åkesson's election speech in August was that Islam was "the Nazism and Communism of our time" and must be approached with "the same disgust and much stronger resistance".

"These guys really care about ideology," says Henrik Arnstad, author of research on fascism cited by the prime minister in defence of his outburst. "You can easily find ultra-nationalism in their politics which is incompatible with liberal democracy. They want to exclude certain people from the nation."

Anti-racists and the media have collaborated to expose the party's extremism, revealing that members had posted racist comments anonymously on far-right websites and, in one case, posed with a swastika. Those involved quit or were expelled, but these revelations appear to have done little to dent the party's popularity.

Niclas Nilsson, the Sweden Democrats' group leader on Kristianstad council, rails against "the huge number of immigrants" and says the country has "the most irresponsible immigration policy in western Europe", although he is unable to say how many immigrants live in Kristianstad.

"Swedish people don't feel at home any more," he said. "The problem we have is basically with the Muslims. They have difficulty assimilating, so much of their culture is based on Islam." Nilsson acknowledged – but had no explanation for – the fact that the vote for his party is higher in isolated, rural areas, where there are fewer immigrants.

Nilsson fears that a higher birth rate among Muslim immigrants will eventually see them in the majority, with the inevitable dilution of the nation's "Swedishness", although he struggles to define this notion. But his biggest concern is that the mainstream parties might adopt much of the Sweden Democrats' immigration policies: "That could be disastrous for our continued growth," he said.

In Gamlegården, an overwhelmingly immigrant area in northern Kristianstad, there are few white faces among the Iraqis, Syrians and Somalis who have made it their home.

Ayub, 43, a bus driver who came from Somalia in the 1990s, is offended at any suggestion that the Sweden Democrats are not a racist party. "They can't hide what they think. They are against the very existence of people like me and want us out of the country," he says.

As a Swedish citizen, he feels confident that he has the same rights as anyone else. "They can't touch me or my family," he says. "The problems will start if they ever come to power."

The Guardian