Along with the likes of Emil Maurice, Ulrich Graf, and Max Amann, Weber, a bouncer at a bar, was among the earliest political associates of Adolf Hitler. Ever ready for a fight, Weber carried a riding crop with him, a habit shared by Hitler in those early years. Otto Strasser denounced Weber as an "ape-like creature" and "the most despicable of Hitler's underlings"; Strasser later claimed that Weber was a pimp at this time.

In late 1921, Weber was one of Hitler's cohorts when the Nazis attacked a meeting of the Bavarian League. Hitler personally beat up the League's leader Otto Ballerstedt, an event that led to him serving a month in prison. At some stage before 1923, Weber lost an eye and often wore a specially made pair of glasses as a result.

Following the 1923 Beer Hall Putsch, Weber, by then a horse trader, was owed $1000 by Hitler after he had bought the debt from Ernst Hanfstaengl. Weber insisted on Hitler paying the debt. The two however remained close and Hanfstaengl later claimed that Weber was one of the few who could make fun of Mein Kampf in Hitler's company.

A city councilman in Munich, he was effectively the boss of the city following the Nazi seizure of power in 1933. Weber became a hated figure in the city, particularly amongst the middle classes, and he became a by-word for corruption as it was regularly questioned how a former hotel bellboy had come to own a number of hotels, villas, petrol stations, a brewery, the city's racecourse, its bus service, and a home in the Munich Residenz. Other titles that he acquired included presidency of the Reichsjagdmuseum and the League of German Riding Stable Owners.

In 1934, during the Night of the Long Knives, Weber was amongst the SS men who travelled to Bad Wiessee to purge the Sturmabteilung (SA) leadership. Hitler personally rewarded him for his involvement by promoting him to the rank of SS-Oberführer. From 1936 to 1939, Weber organized the notorious "Night of the Amazons" carnivals at the Nymphenburg Palace, which featured parades of topless variety show girls dressed only in skin-colored panties.

Ever on the lookout for a chance to enrich himself, Weber was active on Kristallnacht when he took a group of SS men, including a young Hermann Fegelein, to Planegg where they ransacked the estate of Jewish nobleman Baron Rudolf von Hirsch . The estate eventually passed into Weber's possession.

Weber took care of security arrangements for Nazi functions in Munich although he received criticism for this when his plans failed to prevent Georg Elser's bomb attack on the Bürgerbräukeller on 8 November 1939, which missed Hitler and a number of other leading Nazis including Heinrich Himmler and Alfred Rosenberg by only thirteen minutes. Despite this, Weber remained important in Munich, although Gauleiter Paul Giesler was a rival. The two clashed in 1943 over the continuation of horse racing in the city and the dispute was ultimately brought to Hitler himself, where Giesler argued it should be banned as it was not conducive to total war. Hitler agreed in principle with Giesler but, due to the respect for his Alter Kämpfer comrade, he allowed racing to continue at the Theresienwiese only.

Weber died in 1945 after being arrested by the United States Army near Starnberg. He was one of a number of prisoners being carried in an open-backed lorry, which overturned. Weber suffered fatal injuries in the accident. His body was interred in a mass grave at Heilbronn.