In 1947, decorated war veteran Lucas “Luke” Jackson (Paul Newman), is arrested for cutting parking meters off their poles one drunken night. He is sentenced to two years in prison and sent to a Florida prison chain gang run by a stern warden, the Captain (Strother Martin), and a taciturn rifleman, Walking Boss Godfrey (Morgan Woodward), whose eyes are always covered by a pair of mirrored sunglasses. Carr (Clifton James) the floorwalker, tells the rules to the new set of prisoners. Even trivial violations result in a night in “the box”, a small square room with limited air and very little room to move.
Luke refuses to observe the established pecking order among the prisoners and quickly runs afoul of the prisoners’ leader, Dragline (George Kennedy). When the pair have a boxing match, the prisoners and guards watch with interest. Luke is severely outmatched by his larger opponent but refuses to acquiesce. Eventually, Dragline refuses to continue the fight but Luke’s tenacity earns the prisoners’ respect and draws the attention of the guards. He later wins a poker game by bluffing with a hand worth nothing. Luke comments that “sometimes, nothing can be a real cool hand”, prompting Dragline to nickname him “Cool Hand Luke”.
After a visit from his sick mother, Arletta (Jo Van Fleet), Luke becomes more optimistic about his situation. He continually confronts the Captain and the guards, and his sense of humor and independence prove to be both contagious and inspiring to the other prisoners. Luke’s struggle for supremacy peaks when he leads a work crew in a seemingly impossible but successful effort to complete a road-paving job in less than one day. The other prisoners start to idolize him after he makes and wins a spur-of-the moment bet that he can eat fifty hard-boiled eggs in one hour.
One day, Luke picks up a deadly rattlesnake from the grassy ditch and holds it up for Boss Godfrey to shoot with his rifle. Luke tosses the snake to the boss as a joke, before he hands him his walking cane. Dragline advises Luke to be more careful about his actions pertaining to the “man with no eyes.” A rainstorm causes everyone to prematurely end their work. Before he joins the other prisoners in the truck, Luke shouts to God, testing him. On that same evening, Luke receives a letter stating that his mother has died.
The Captain anticipates that Luke might attempt to escape in order to attend his mother’s funeral and has him locked in the box. After being released from the box, Luke is told to forget about his mother now that her burial is completed but he becomes determined to escape. Under the cover of a Fourth of July celebration, he makes his initial escape attempt. He is recaptured by local police and returned to the chain gang, but one of the bloodhounds sent after him dies from heat and overexertion. The Captain has Luke fitted with leg-irons and delivers a warning speech to the other inmates, explaining, “What we’ve got here is failure to communicate. Some men you just can’t reach. So you get what we had here last week, which is the way he wants it. Well, he gets it. And I don’t like it any more than you men.”
A short time later, Luke escapes again by deceiving the guards while taking a break to urinate, and removes his shackles with an axe at a nearby house. He spreads curry powder and chili powder across the ground to keep the guard dogs from following his scent. While free, Luke mails Dragline a magazine that includes a photograph of himself with two beautiful women. He is soon recaptured, beaten, returned to the prison camp, and fitted with two sets of leg irons. Luke is warned by the Captain that if he ever attempts to escape again, he will be killed on the spot.
Luke becomes annoyed by the other prisoners fawning over the magazine photo and reveals it to be a fake. At first, the other prisoners are angry, but when Luke returns after a long stay in the box and is punished by being forced to eat a massive serving of rice, the others help him finish it.
As further punishment for his escape, he is forced to repeatedly dig a grave-sized hole in the prison camp yard, fill it back in, and is then beaten. The prisoners observe his persecution, singing spirituals. Finally, as the other prisoners watch from the windows of the bunkhouse, an exhausted Luke collapses in the hole, begging God for mercy and pleads with the bosses not to hit him again. Believing Luke is finally broken, the Captain stops the punishment. Boss Paul warns Luke that he will be killed if ever he runs away again, which Luke promises in tears not to do. The prisoners begin to lose their idealized image of Luke, and one tears up the photograph of Luke with the women.
Working on the chain gang again, seemingly broken, Luke stops working to give water to a prisoner. Watched by the disappointed prisoners, he runs to one of the trucks to take Boss Godfrey’s rifle to him. After Boss Godfrey shoots a snapping turtle, Luke retrieves it from a slough for him, complimenting the boss for his shot. Luke is ordered to take the turtle to the truck but steals the dump truck and the keys to the other trucks. In the excitement of the moment, Dragline jumps in the dump truck and joins Luke in his escape. After abandoning the truck Luke tells Dragline that they should part ways. Dragline reluctantly agrees and leaves. Luke enters a church, where he talks to God, whom Luke blames for sabotaging him so he can’t win in life. Moments later, police cars arrive. Dragline walks in and tells Luke that the police and bosses have promised not to hurt Luke if he surrenders peacefully.
Instead, Luke opens a window facing the police and mocks the Captain by repeating the other man’s earlier speech (“What we’ve got here is a failure to communicate”). He is shot in the neck by Boss Godfrey. Dragline carries Luke outside and surrenders, but charges at Boss Godfrey and strangles him until he is beaten and subdued by the guards. While Luke is loaded into the Captain’s car, Dragline tearfully implores him to live.
Against the protests of the local police, the Captain decides to take Luke to the distant prison infirmary instead of the local hospital, ensuring Luke will not survive the trip. As the car drives away, a semi-conscious Luke weakly smiles while the tires crush Boss Godfrey’s glasses. After Luke’s implied death, Dragline and the other prisoners fondly reminisce about him.
In the final scene, the prison crew is seen working near a rural intersection close to where Luke was shot. Dragline is now wearing leg irons, and there is a new Walking Boss. As the camera zooms out, the torn photograph of Luke grinning with the two women has been taped back together and is superimposed on a bird’s eye view of the cross-shaped road junction.