Revealing mistake: After a shootout with Baby Face Nelson at a hotel, Melvin Purvis converses with J. Edgar Hoover over the telephone. As he says, "Our type cannot get the job done." Watch the background. You can see a modern era train move along the tracks in the darkness. This is inconsistent with the film's setting of 1933/1934.
Revealing mistake: In an aerial shot of Lincoln Ave in Chicago outside the Biograph Theater the rubber mats laid on the road to look like 1930's brick streets only cover the middle 1/3rd of the street. On the outside 2/3s the street is modern asphalt and shows striping tape for bike lanes.
Revealing mistake: Just after the Little Bohemia scene when Melvin Purvis is shooting Baby Face Nelson, you can see a whole bunch of flashes going off in Nelson's jacket, which are obviously charges going off.
Revealing mistake: In the scene where Dillinger breaks out of jail in Indiana, there is a shot of the getaway car and its license plate, and you can see that the license plate says "Historic Vehicle" on it.
Chosen answer: Baby Face Nelson doesn't pick up Dillinger. He picks up the characters played by Stephen Dorf and another associate. Dillinger and Hamilton escape through the woods and steal a car from an older man they come across in the forest. After the shootout, Dillinger and Nelson are never in the car together.