Jurgen Klopp's Premier League leaders were stunningly held to a 2-2 draw by the third-tier club on Sunday.
The Reds will be expected to come through the replay at Anfield but face a tricky test in the last 16 should they do so in the form of a trip to Stamford Bridge to face Frank Lampard's Blues.
Holders Manchester City were handed an away tie at Championship promotion chasers Sheffield Wednesday, while Manchester United could renew acquaintances with record goalscorer Wayne Rooney if his Derby County side defeat fourth-tier Northampton Town in a replay.
There will be an all-Premier League tie regardless of who comes through the replay between Tottenham and Southampton, the winner set to host relegation-threatened Norwich City.
Arsenal, who have won the competition a record 13 times, can secure their place in the fifth round on Monday with victory at Bournemouth, whose south-coast neighbours Portsmouth await the victor.
The fifth-round ties will be played from March 2-5 with no replays.
— The Emirates FA Cup (@EmiratesFACup) January 27, 2020
Draw in full:
Sheffield Wednesday v Manchester City
Reading/Cardiff City v Sheffield United
Chelsea v Shrewsbury Town/Liverpool
West Brom v Newcastle United/Oxford United
Leicester City v Coventry City/Birmingham City
Northampton Town/Derby County v Manchester United
Southampton/Tottenham v Norwich City
Portsmouth v Bournemouth/Arsenal
Photo Getty Images Caption: Liverpool manager Jurgen Klopp