A more energetic United display looked like going unrewarded against a dogged Crystal Palace outfit until midfielder Fred struck first-time from Mason Greenwood's lay-off to send a superb 20-yard effort curling over Vicente Guaita and into the top corner for the winner.
It was only Fred's second goal of the season - and it was the first time since the middle of September that United have recorded back-to-back league wins.
Until the Brazilian's effort, Rangnick's side created numerous decent chances, without really looking as though they were going to score and Guaita would probably have saved an Alex Telles free-kick that skimmed the bar had it been on target.
"I am very happy with the way the team performed, especially the first half hour, with the pressing, it was exceptional," Rangnick told BBC Match of the Day. "The only thing missing was the 1-0 or 2-0.
"The way we defended, we had control of the whole game.
"The clean sheet was the most important part. These are the things we must improve. We need to keep clean sheets. With just the one training session, I was really impressed. We did much better than expected.
"We always tried to be on the front foot. We were never not apart from maybe the last five minutes. At all other times we were trying to keep them away from our goal."
Palace, who were looking to win for a third successive season at Old Trafford, came close to taking the lead themselves when Jordan Ayew turned James Tomkins header across the face of goal and narrowly wide of the far post.
The victory meant that Rangnick emulated three of United's other four new managers since Sir Alex Ferguson retired in 2013 by winning his first match at the helm, with Louis van Gaal the only man to fail.
Follow live text and radio commentary of Manchester United v Crystal Palace, plus the rest of Sunday's Premier League matches
Go straight to all the best United content
Visit our Palace page
Fred the enigma turns match-winner
Ralf Rangnick spent most of the match standing at the edge of his technical area
Fred was one of the major focuses for criticism during Ole Gunnar Solskjaer's reign as manager, which ended last month.
As Michael Carrick pointed out during his three-match spell as United's caretaker manager, the midfielder is a mainstay of a Brazil team that dominated World Cup qualification in South America.
Yet he has never given any indication, through either goalscoring or chance creation, that he is worthy of the £47m United paid for him.
He is industrious, though. And, judging by his little dance after the celebrations of his goal broke up, he is not without personality, which may be helpful to Rangnick over the coming months.
The German has not had a full training session with his players yet but there was certainly an intensity about United's early pressing, with Cristiano Ronaldo among those doing it on a couple of occasions.
It did harry Palace into a couple of mistakes - but then, as happened so many times under Solskjaer, the fire went out of the hosts and they started to rely more on crosses and individual moments to cause the visitors problems.
Marc Guehi managed to block a Ronaldo volley at close-range after the Portuguese had beaten the Palace offside trap and Tomkins headed a curling Fred shot away from danger.
Probably the hosts' best opportunity in the opening period fell to Bruno Fernandes, who didn't hit his shot cleanly but still forced a low full-length save out of Guaita after Ronaldo had set up the chance with an elaborate header.
United did not reach the same heights after the interval, although the introductions of Greenwood and Anthony Elanga at least kept them pushing forward until the decisive moment came.
"I had to ask my assistant coach if that was Fred's right foot," added Rangnick. "I thought he could only shoot with his left. I'm happy for him.
"It's always easier to build on things after you're successful. We must produce further clean sheets and get better at creating chances for our attack. Overall, I was very pleased with the way we played today."
Another defeat for Palace
Manchester United 1-0 Crystal Palace: Patrick Vieira 'disappointed' with result
Palace have now lost three Premier League games in a row for the first time under Patrick Vieira's management.
The Eagles are six points above the bottom three and clearly need to halt the slide before they end up in trouble.
Having achieved their previous victories in this stadium with 29% and 24% possession respectively, Palace actually did far better in this contest, with 39%.
But, other than Ayew's effort, an early Wilfried Zaha shot easily saved by David de Gea and a flurry of late set-pieces sent into the United box that were all cleared, Palace lacked conviction.
Zaha was a particular disappointment as he hardly got into the game, especially in the second-half when Cheikhou Kouyate and Conor Gallagher carried a greater threat.
"I'm disappointed - we gave them a tough game, we made it really difficult for them," manager Patrick Vieira told Match of the Day.
"At the end, conceding the goal, we are so disappointed. And we had the chance before [for Ayew], it is frustrating.
"When you get opportunities, you have to take them, and when you don't take them your are left vulnerable. I'm am disappointed for Jordan as he had a fantastic game. That is football.
"We need to change our mindset. How can we be more ruthless when we're in the opposition's box? That's something we have to work on."
- For a man whose footballing epiphany came almost 40 years ago and continues to underpin his style of play, Ralf Rangnick maintains a thirst for innovation
- Our swimming school forms groups of 6-10 people, which is the best option for learning. In such groups
- Beginners and seasoned players alike can learn the game and earn some extra cash while doing so. A live dealer also allows players
- England spinner Jack Leach should not have been selected for the first Ashes Test and has been given "a hospital pass", says Phil Tufnell.