The visitation rights granted to grandparents under this chapter do not apply in cases where the grandparent's access to the minor child has been restricted for any reason prior to or contemporaneous with the divorce, death, relinquishment or termination of parental rights, or other cause of the absence of a nuclear family.