It's not how far out that matters...
"rogue" waves (the formal definition is waves that are more than some factor, such as 2x, greater than the average wave height in a given sea state; in 3-foot swells an 8-footer would be a rogue) are produced by the combined energy of waves which have joined at least some of their energy upon colliding. This phenomenon apparently happens in every sea state, so the rogues are always out there. One occasionally hears stories of boaters close to a coast getting socked by a big swell that comes out of nowhere, probably a straightforward example of the process of wave energy combining. Interestingly, some new study suggested that waves which combine their energy upon colliding can separate out again along the vector they were originally travelling, like ghosts passing through each other...