Pessimism is "most pronounced for those with at most a high school education," according to the New York Federal Reserve.

"Since the beginning of the year, the economy has created a paltry 600k jobs, but without the job growth in these industries, there would be zero job growth."

In the latest sign of trouble for the U.S. labor market, confidence in the ability to move from one job to another has hit a record low.