The Black Stars, who lost to Group G rivals South Africa last month, won 3-1 in Cape Coast thanks to goals from Mohammed Kudus, Thomas Partey and Andre Ayew.
Ghana are on six points, one behind leaders South Africa after Bafana Bafana beat Ethiopia earlier on Saturday.
Elsewhere, Senegal and Morocco extended their 100% starts to strengthen their positions in Group H and Group I respectively.
Only the 10 group winners progress to next March’s play-offs to decide the continent’s five representatives in Qatar.
Black Stars keep pace with Bafana Bafana
Rajevac led Ghana to the quarter-finals of the 2010 World Cup and the Serb started his second stint with a vital three points following his re-appointment last month on a one-year contract.
Kudus put the Black Stars in front with a low shot from just outside the box in the fifth minute but Zimbabwe captain Knowledge Musona beat debutant goalkeeper Jojo Wollacott from the penalty spot to equalise four minutes into the second half.
Arsenal midfielder Partey fired into the bottom left-hand corner to make it 2-1 in the 66th minute and Ayew’s downward header wrapped up the scoring with three minutes left.
A home victory had been crucial for Ghana after South Africa had beaten Ethiopia 3-1 in Bahir Dar.
Home goalkeeper Fasil Gebremikael spilled Teboho Mokoena’s free-kick in first-half stoppage time to give South Africa the lead at the break.
Getaneh Kebede levelled for Ethiopia midway through the second period with a fine free-kick but Mothobi Mvala put Bafana Bafana back in front from a long throw with 19 minutes remaining.
Evidence Makgopa lobbed the keeper to make the game safe for the visitors in added time after coming off the bench.
Hugo Broos’ side, who failed to qualify for the Africa Cup of Nations, host Ethiopia on Tuesday while Ghana travel to Harare.
Morocco march on as Guinea draw again
Elsewhere, Morocco eased to a 3-0 victory over Guinea-Bissau in Casablanca.
Ayoub El Kaabi and Aymen Barkok put the Atlas Lions 2-0 up at the break, and El Kaabi grabbed his second of the game with 20 minutes remaining.
Morocco had beaten Guinea-Bissau 5-0 on Wednesday and Vahid Halilhodzic’s men now have nine points from three games in Group I.
Meanwhile, Guinea’s hopes of progressing to the play-offs took another blow as they were held to a 2-2 draw by Sudan, who can no longer qualify.
In a match played in Agadir, Morocco, following the coup in Guinea last month, Jose Kante put Syli National ahead with a fierce left-footed strike three minutes into the second half.
Substitute Ahmed Hamid equalised for the Sudanese on 64 mins but Guinea restored their lead three minutes later when Mohamed Bayo slotted in from a one-on-one.
But defender Amir Kamal headed in a free-kick to net a second equaliser for Sudan with two minutes left, and secure a second draw between the sides in the space of four days.
All three of Guinea’s matches have ended in stalemate and Didier Six’s side realistically need to beat Morocco when the two sides meet in Agadir on Tuesday to have a chance of claiming top spot.
Mane on scoresheet for Senegal
In Group H, Senegal beat closest challengers Namibia 4-1 in Thies to establish a five-point cushion over the Brave Warriors.
Idrissa Gueye and Famara Diedhiou netted in the first half for the Lions of Teranga, and Liverpool forward Sadio Mane added a third 10 minutes after the restart.
Joslin Kamatuka pulled one back for the visitors with 15 minutes remaining but Keita Balde restored the three-goal cushion in the 84th minute.
Senegal have maximum points from three games, and face Namibia in Johannesburg on Tuesday.
Meanwhile, Togo and Congo drew 1-1 to leave both countries without a win after three matches and facing elimination.
An own goal by Alaixys Romao gave Congo the lead at the break in Lome, but Euloge Placca Fessou struck a powerful half-volley to equalise 11 minutes after the restart.
Congo’s Guy Mbenza hit the bar with a header but the Red Devils had to settle for a point and then had goalkeeper Christoffer Mafoumbi sent off after the final whistle.