The ending of The New Mutants says that the Demon Bear is Dani's own fear, and she ends it by facing that fear instead of fighting it. In my opinion, that is the cleanest idea in the film, because it ties the creature that wrecked her home to the power that pulls fears out of other people.
The chain of scenes matters here. The group finds evidence that Reyes works for a corporation and is training them as weapons, not heroes, and then confronts her. Reyes tries to suffocate Dani, and that attack is what unleashes the Demon Bear, which kills Reyes. So the villain of the film is not the creature alone: the adult who confined the teens is the one who provokes it. It could be argued that this makes the monster almost a victim of the hospital too, since it is made from the same terror Dani carried in from the start.
The visit from her dead father is the hinge. He urges her to face her fear instead of running, and she calms the Demon Bear rather than beating it. That is a quieter climax than the cosmic showdowns of X-Men: Apocalypse or Dark Phoenix, where Jean Grey absorbs a huge cosmic energy force and the finale is a battle on that scale. Here the stakes stay inside one locked ward and one frightened girl. The fight is not left to her alone, since Illyana brings her Soulsword and armor from her own magical realm and the others hold the line. That detail pays off the early picture of Illyana as hostile and mistrustful, because she is the one who steps up when it counts.
The production context supports this reading. Producer Karen Rosenfelt described the film as a psychological horror story, and then Fox chief Stacey Snider compared it to One Flew Over the Cuckoo's Nest and The Shining, according to the Boston Globe. Rosenfelt also called the Medfield hospital 'a character in and of itself'. A story set in a real historic hospital, where the monsters are memories and fears, resolves best by calming a fear rather than staging a punch up. Compare Logan, another spinoff that ends on a small, personal scale: both films choose intimacy over spectacle, though Logan ends on a farewell and this one does not.
With Reyes dead, the hospital no longer holds them, and the five teens walk out at sunrise. It could be argued that the corporation behind Reyes is left as an open thread, because the film shows no further story. I read the closing image as a team that has only just formed, which is a promise of more rather than a goodbye.
Does that make a follow up likely? Nothing on screen settles it. The new X-Men film in development is a separate Marvel Studios project, covered on the new X-Men movie page. For where this ending lands among its siblings, see X-Men movies ranked.