He attended Harding Academy of Memphis in Memphis, Tennessee. He was selected to the 2009 USA Today All-USA Second Team. He also was selected to the 2008 Atlanta Journal-Constitution Super Southern 100. He was named to the 2007 Tennessee Sports Writers Association Division 2-A All-State.
In 2009, he was a recipient of the Vickie and Leon Farmer Football Scholarship. In 2010, he was selected as team’s Most Improved Player Award.
On April 28, 2013, Brown signed with the Baltimore Ravens as an undrafted free agent following the 2013 NFL Draft. Brown was one of the 2 rookie free agents to make the 53-man opening day roster. Brown made his NFL debut against the Denver Broncos on September 5, 2013. In that game, Brown recorded his first-career reception and his first-career touchdown. In Week 14 of the 2013 season, he caught the game-winning touchdown from quarterback Joe Flacco with 4 seconds remaining in regulation for a 29-26 victory over the Minnesota Vikings; Brown won NFL Rookie of the Week honors for his performance. Brown led the Ravens in receiving touchdowns and tied the Ravens rookie record for receiving touchdowns in a season with seven, achieving that record against the Cincinnati Bengals; Brown also became the first rookie in Ravens franchise history to score a TD in each of his first two career games. He finished the season with 49 receptions for 524 yards, which ranked 3rd and 2nd on the team, respectively.