The victory gave India six points from two matches, making them the only unbeaten team in the pool following England's surprise 4-1 win over Olympic champions Germany earlier in the day.
New Zealand took the initial lead in the third minute when India conceded a self-goal, which has been introduced into world level hockey after the Olympic Games.
Rupinderpal Singh failed to cleanly stop Sardar Singh's back-pass in front of the goal and the ball trickled over the goal-line to concede India's first self-goal in international hockey.
New Zealand defence also wilted under pressure to give India the equaliser through a self-goal.
Akashdeep Singh dived for a deflection on Sardar long through ball, which went into the goal off a defender to give India the equaliser in the 10th minute.
A fine field goal by Gurwinder Singh Chandi (14th) and a penalty corner conversion by V R Raghunath (25th) gave India a 3-1 lead at the interval before New Zealand came back through a flash strike by Nicholas Wilson two minutes into the second half.
Mounting pressure through repeated raids, India put the game out of New Zealand's reach when Danish Mujtaba capitalised on a rebound from the goalkeeper on a pressing move in the 65th minute.
This was India's second win in successive days, following the 3-1 win over England in their opening match yesterday.
India coach Michael Nobbs was delighted at the way the new-look team had produced two victories.
"I'm happy with the boys and the manner in which they have played," Nobbs said.
"They are a young bunch and they've combined well in their first tournament together.
These two wins will boost the confidence of the youngsters a great deal," he added.
India next play Germany on Tuesday after a rest day tomorrow.
Sardar was under pressure from the New Zealand strikers when he sent a diagonal back-pass to Rupinderpal inside the circle.
Rupinderpal's inability to cleanly trap the ball conceded the early lead, but India did well to recover from the lapse.
Mounting pressure through a couple of fine raids got India back into the game, and Sardar's ability to find the gaps through long balls caused the trouble that got India the equaliser.
Akashdeep dived to find the though-pass from the midfield.
Four minutes later, Gurwinder shot in from close in a breakaway move down the right flank and India were on a roll, although they conceded two penalty corners before forcing one at the other end.
New Zealand's Stephen Jenness had the Indian goal under pressure when he trapped a cross from the right inside the circle and unleashed a firm reverse shot that was blocked by Indian goalkeeper T R Potunuri.
Indian defender Manpreet Singh then made the save of the match on New Zealand's second penalty corner.
Standing on the goal-line, he swung away the 'ball off a rising drag-flick'.
At the other end, Raghunath converted India's first penalty corner with a rasping straight drag-flick that went into the net to give India a 3-1 lead at halftime.
India's lead was reduced within two minutes of resumption when Wilson sent a smashing hit into the boards.
India then came out to mount a series of raids that repeatedly saw the ball go into the rival circle, but they failed to get it past New Zealand goalkeeper Devon Manchester, who often came out to narrow the angle of the Indian strikers.
Akashdeep missed a sitter in the 55th minute when he sent in a feeble shot straight into the goalkeeper's bads after receiving a pass from Chandi on top of the circle.
Sardar again shaped the move that got India their fourth goal, sending in long ball to S K Uthappa, who moved into the circle but is shot was blocked by the custodian.
The rebound went to Mujtaba, who made no mistake in smashing it into the boards.
Manpreet was struck on the face on the follow-through of a New Zealand player's stick and had to leave the pitch two minutes before the hooter.