The majority of physicists say time travel probably won't work (at least in the Hollywood-go-anytime-you-wanna sense). Several time traveler parties have gone famously unattended by time travelers (at least, any willing to fess up about it). In general, science is kind of a buzz-kill on this one.
But if you want some justification for your daydreams, the person to talk to is Ronald Mallett, a theoretical physicists at the University of Connecticut who is most well-known for being the guy who thinks time travel is totally possible. (Mark wrote about him here back in 2007.) In fact, in 2006, Mallett predicted that time travel would be figured out within a decade.
I honestly have not researched this enough to give you my opinion on Mallett's ideas. His fellow physicists have addressed it, though. You can read one response to Mallett at arXiv. All of that is a long, context-relevant introduction to the video above, where Mallett explains his theories. I wanted to post the video because it's interesting and I thought you all would dig it. I'm also interested in the new video series this comes from—EPIPHANY, a daily video about big ideas taken from interviews with journalists, tech thinkers, scientists, and more. Mallett makes an interesting kick off for a series like this.
My hope, though, is that EPIPHANY doesn't only focus on scientific ideas that are kind of on the fringe. There's so many amazing discoveries that have the bulk of evidence behind them, it seems like a waste of a good platform to not cover the stuff that's more likely to be true.
Check out the rest of the EPIPHANY videos. So far, the series includes clips from interviews with Ronald Mallett, Reddit founder Alexis Ohanian, and filmmaker Jason Silva.