Make a point of both planning and carrying out whatever is necessary to get your kids fishing. While it's tough and sometimes even a frustrating pain when they're little, you're getting life-long fishing buddies of the best kind out of the deal. Fast forward a couple of decades, and you're the one holding them back. You may have to trade some quality time on trophy streams for some days with snotty noses, worms, and bluegills, but you won't be lying on your deathbed someday, wishing that you'd ignored your kids more so that you could have caught another trophy fish.
My youngest is 21 and loves to fish. He couldn't have cared less for all the years I traveled. The older son was really into it and grew 'out' of it when he hit his early teens.
I spent more quality time with my dad on fishing trips than anything else. It's a great way to teach kids many things and spend good time with them.