One year after graduation, New York City lawyer James “Brick” Davis (James Cagney) has no clients because he refuses to compromise with his ideals and integrity. His friend Eddie Buchanan (Regis Toomey) tries to recruit him as a federal agent or “G Man” (government man), but Davis is unsure. However, when Buchanan is killed while trying to arrest a gangster, Davis changes his mind, determined to bring the killer to justice. He bids farewell to his mentor, “Mac” MacKay (William Harrigan), a mob boss who financed his education to keep Davis on the right side of the law. He bids farewell to Jean Morgan (Ann Dvorak), the star of MacKay’s nightclub who has feelings for Davis.
Davis travels to Washington, D.C. to begin his training. A mutual dislike forms immediately between him and his instructor, Jeff McCord (Robert Armstrong) which eventually subsides as time passes, but not before McCord openly mocks and derides Davis’ attempts at training . However, Davis is attracted to McCord’s sister Kay (Margaret Lindsay) which strengthens his determination to remain passive despite McCord’s efforts to rile him.
Meanwhile, MacKay retires and buys a resort lodge out in the woods of Wisconsin. His men, free of his restraint, embark on a crime spree. Hamstrung by existing laws (federal agents have to get local warrants and are not even allowed to carry guns), the head of the G-Men pleads for new laws to empower his beleaguered men. They are enacted with great speed.
Davis identifies one of the perpetrators, Danny Leggett (Edward Pawley), by his superstition of always wearing a gardenia. Not having completed his training, he can only give agent Hugh Farrell (Lloyd Nolan) tips on Leggett’s habits. Farrell tracks down and arrests his quarry, but he and some of his men are gunned down, and Leggett escapes.
McCord is put in charge of the manhunt and given his choice of five agents. He picks Davis, a decision that later pays dividends when Jean is brought in for questioning, Davis learns she is now married to Collins (Barton MacLane), one of the crooks. She inadvertently lets slip that the gang is hiding out at MacKay’s lodge (against MacKay’s will). In the ensuing wild shootout, Davis kills MacKay, who was being used as a human shield. Before he dies, MacKay forgives his distraught friend. Davis then tries to resign from the department but McCord talks him out of it by reminding him that McKay’s death wasn’t his fault and asks him to stay on.
Only Collins gets away. McCord and Davis go to Jean’s apartment to warn her. Jean is not there, but Collins is, and shoots at them. Davis pushes McCord out of the way and takes a bullet meant for him. Collins gets away. Davis ends up in the hospital (where Kay is a nurse) for his shoulder wound. Collins kidnaps Kay to use as a hostage. Jean finds out where he is hiding and telephones Davis, only to be shot by her husband. Davis bolts from his hospital bed, has some final words for the dying Jean, sneaks inside the garage and rescues Kay. Collins is shot to death by McCord as he tries to drive away. Kay escorts the still-bandaged Davis back to the hospital, vowing to “handle your case personally.”