It would definitely be possible for them to bend reality in such a way that they could trap the invisible character, such as creating mazes where only they can see the real path, hence trapping him, etc.