One would think that the idol worship of the calf would be more serious than being panicking and attempting to run away when being faced by war. Rav Kahn points out that the Chet Ha'aigel was not really a rebellion against Hashem, but was an attempt to replace the missing Moshe Rabbeinu (whom they thought was dead after 40 days on the mountain) with another representative. On the other hand, the Chet Hameraglim was a complete rebellion against Hashem and an attempt to totally reject Hashem's purpose for Bnei Yisroel.
Another aspect actually ties into the reason that Avraham Avinu sent Eliezer to his family to get Rivkah as a wife for Yitzchak rather than take a woman from the Canaanim. Both groups were idol worshipers. However, the family of Bethuel were idol worshipers because of an error in analysis. The Canaanim were idol worshipers because of a basic flaw in their makeup.
Similarly, the Chet Ha'aigel was an error in analysis. Only 3,000 out of 600,000 actually worshiped the aigel and were punished by death. The rest fell prey to the mistake that caused them to believe that they were just setting up a representation of Hashem to help them. Their bsic desire was to be closer to Hashem and follow what they thought was His will. The meraglim on the other hand fell prey to a basic flaw in character. They insisted on putting their own wishes, desires, and fears ahead of the explicitly stated commands of Hashem. We see this in a number of places in the story. From the very beginning when they said that "we appeared to them as grasshoppers and so we were in our own eyes", to the end when the "acknowledged" their mistake by attempting to go up an conquer Eretz Canaan even though Hashem told them not to. Even their "atonement" was actually a rebellion against Hashem.
Rabbi Kahn points out that the difference between the two sins is also shown in the tragedies of the fast days that were instituted on those days. The Chet Ha'aigel occurred on Shiv'a Asar B'Tamuz, while the Chet Hameraglim was on Tish'a B'Av. It was the Chet Hameraglim that triggered the final destruction and the full mourning. The tragedies that occurred on Shiv'a Asar B'Tamuz were things that could be forgiven and recovered from. The destruction of Tish'a B'Av was final and could only be endured. As Rabbi Kahn says
The reason that the fast of Tisha Be-Av is observed in such a unique way is because it does not commemorate a calamity that can be removed through prayer and repentance; it is rooted in historical events that led to a gezeira (decree) that could not be changed. The meraglim episode led to the decree that the generation would perish in the wilderness. Later in history, the Temples were destroyed on Tisha Be-Av. Prayer and repentance could no longer prevent the destruction. It already was too late.As opposed to calamity, a gezera cannot be removed. It expresses not Divine Providence, but rather the distancing of the Divine Presence and God "hiding His face," as it were. "R. Elazar said: Since the day on which the Temple was destroyed, there is a wall of iron that stands between Yisrael and their Father in Heaven" (Berakhot 32b). The reaction to a gezera is not prayer, but rather mourning and surrender to God's inscrutable will: "And R. Elazar said: Since the day on which the Temple was destroyed, the gates of prayer are locked" (ibid.).The seventeenth of Tammuz, despite the five tragic events which took place on that day, is defined as a day of calamity. It is true that on this date the first set of tablets were shattered, but following prayer on the part of Moshe Rabbeinu and teshuva on the part of the nation, we merited to receive the second luchot. On this date, the walls of Jerusalem were indeed breached, and the enemies stood ready to enter, and it was therefore a time of calamity for the Jewish nation, since the destruction had not yet occurred. But on Tisha Be-Av, a tragic decree had already been issued. Despite Moshe's entreaties, the attempts to mitigate the sharpness of the decree, the attempt to repent and continue on to Eretz Yisrael was futile and reached its tragic conclusion at Chorma (Bamidbar 14:45).