“So as with the other covenants, it is possible for someone to join the new covenant community externally without the new heart that defines that covenant. He may be baptized and profess Christian doctrine. But if he lives a life of sin, he shows that he does not have the new heart that is the mark of the new covenant. He has wrongly entered the covenant community and ought to be disciplined by the body. He has become a Christian externally, but without inward change.”

John Frame, Systematic Theology, p. 81.

I would add that we should read “wrongly entered the covenant community” as “entered the covenant community deficiently” not that he shouldn’t have entered the covenant community at all. Baptism welcomes us as legitimate members of the covenant community – but, in the case of an unconverted covenant member, his membership is belied by his lack of a new heart. But it wasn’t wrong to baptize him and admit him to the covenant community – for it is not our place to read the heart but to evaluate words and actions to the best of our ability.