I think just about everyone considers the era after the 1973 oil crisis as one of economic decline in the West, as well as many developing countries.
The stagnation is mostly called that because of political/cultural factors, as nothing really changed in that - politics were the same every year, very few cultural products were allowed past censors, or were so watered down by them that they lost their substance. It's true that many people in the USSR achieved a quality of life comparable to that of Western and Northern Europe at the time, whereas before they didn't have shit. But economically, the state focused on megaprojects as an engine for economic growth, which ended up being costly multi-decade boondoggles.
These projects were also focused mostly on natural resource extraction, while in areas like computers (and really, most consumer goods as well), the Soviet Union pursued a strategy of developing analogues of Western products, and for that reason failed, as the Americans were developing new integrated circuits while the Soviets were still copying the old ones. And not only that, but putting their best minds to copying instead of developing something new, and as a result falling farther and farther behind, as development of electronics accelerated.
In terms of military spending, actual estimates differ widely because dual use technologies poorly implemented in the civilian sphere are often counted as a purely military pursuit. However, from what I understand, it was actually manageable:
Shlykov wrote:Gosplan's official data, for example, show that in the late 1980s, the defense industry employed 9.5 mln people (including 6.5 mln to 7 mln in the Russian Federation) of the total 130 mln workforce, consuming 20% of sheet steel, 9.3% of rolled steel and 23.6% of rolled aluminum products, while the complex's fixed productive capital was 6.4% of the Soviet aggregate.
Another problem that is often left out is that the sophistication of the Soviet economy was such that a centralized planning model could no longer handle it, even in a simplified model as implemented under the Kosygin/Liberman reform. The problem was that there were so many competing stakeholders and interests that their competition for administrative resource hurt the economy as a whole. For example, the Moskvitch plant was in constant competition with Lada, but this meant that Lada received favorable treatment in investments and production targets while Moskvitch was forced to produce more cars at the expense of quality and had its model lineups be delayed and cancelled as they ran into bureaucracy at the ministry of automobile industry. Lada itself fell victim to this as in the 80s, the state forced it to spend loads of its own money and resources building an R&D center for cars in general even though there was already a government one near Moscow. There was basically no mechanism to deal with these issues.
The economic model in general also favored capital reinvestment for the sake of capital reinvestment because of what was essentially a socialist overproduction crisis in the extraction industries. This meant that the best factories were those that spent the most money and it didn't matter if this meant higher production efficiency because that was not the goal. This is also what led to product deficits, as more money was spent (read: given to people for their work) but this didn't lead to higher output of consumer goods.
The structural crisis in agriculture is obviously another problem, but one that predates Brezhnev.