Most existing works on cognitive radio networks assume that cognitive (or secondary) users are capable of switching/jumping to any available channel, regardless of the frequency gap between the target and the current channels. Due to hardware limitations, cognitive users can actually jump only so far from where the operating frequency of their current channel is, given an acceptable switching delay that users are typically constrained by. This paper studies the performance of cognitive radio networks with dynamic multichannel access capability, but while considering realistic channel handoff assumptions, where cognitive users can only move/jump to their immediate neighboring channels. Specifically, we consider a cognitive access network with m channels in which a cognitive user, currently using a particular channel, can only switch to one of its k immediate neighboring channels. This set of 2k channels is referred to as the target handoff channel set. We first model this cognitive access network with restricted channel handoff as a continuous-time Markov process, and then analytically derive the forced termination probability of cognitive users. Finally, we validate and analyze our derived results via simulations. Our obtained results show that the forced access termination probability of cognitive users decreases significantly as the number k increases.