There seems to be some correlation with economic conditions. The drop between 1983 (when employment was over 10%) and the mid 1990s is puzzling, but as I've noted in other posts, Americans seemed to be disgruntled in the mid-1990s. The other puzzle is that it hasn't recovered much since the recession of 2008.
The original data was available for some of the surveys, so I broke it down by party identification for those. In the 1980s and 1990s, the supporters of the different parties all had about the same expectations. In the two surveys taken while George W. Bush was president, Republicans were more optimistic; in the surveys taken while Barack Obama was president (counting December 2008 as part of his presidency), Democrats were optimistic. That is, expectations have become polarized by party identification. That could be because of change in communications technology, with supporters of each party increasingly living in "bubbles." However, I think that what has happened since 2008 suggests that something else is also at work. Between January and December 2008, expectations dropped among Republicans and Democrats, which is to be expected given the decline in economic conditions (unemployment rose from 5% to 7.3%). They dropped by more among Republicans, which is reasonable because their candidate had lost in the presidential election. Between December 2008 and October 2010 they dropped sharply among Republicans and stayed about the same among Democrats. By that point, unemployment was even higher (9.4%), although it was dropping, so either response could be seen as reasonable. Between October 2010 and April 2011 expectations dropped sharply again among Republicans and stayed the same among Democrats. Unemployment had fallen slightly (to 9.1%), so it's not clear why expectations should have declined. Finally, the last survey for which a breakdown by party is available was in December 2012, when unemployment had fallen to 7.9%. Democrats became a bit more optimistic, while Republicans remained as pessimistic as ever. That is, during the Obama presidency, expectations kept becoming more negative among Republicans even while conditions were improving--Republicans were far more negative in December 2012 than they had been in December 2008, when there seemed to be a real possibility that we were facing another Great Depression. That is, opinions among Democrats have pretty much followed economic conditions, while opinions among Republicans have diverged from them.
My thought is that this is connected to the rise of ideological conservatism that I discussed in my last post. A lot of conservative intellectuals have become attracted to the idea that we are at a crossroads, with a choice between advancing towards utopia and falling into barbarism (or at least becoming like Western Europe). That mood can trickle down to ordinary people who aren't paying that much attention to politics. As a result, things like Obamacare and raising the debt ceiling, or whatever else was going on in Obama's first term, had as much effect on Republican views as the economic collapse of 2008.
[Data from the Roper Center for Public Opinion Research]