Orange County is located outside of NYC, but it is not so far from the city as to be problematic if you need to commute to Manhattan (or Brooklyn/Queens/The Bronx) for work. There is a commuter train line that runs from Penn Station, across the Hudson River, up through New Jersey, and into Orange County, New York. This train line will make stops in many of the larger towns in Orange County. The advantage to living in Orange County is that the cost of living is less than it is in Manhattan. However, if you're looking to rent an apartment, instead of buying or renting a house, then you will need to know what areas to look at. Some places in Orange County are populated predominantly by single-family homes. There are, however, a few places that do have lots of rental apartment buildings. Here are two areas to consider.
Middletown is one of the larger towns in Orange County. There are around 28,000 residents. The area has tons of shopping, restaurants, professional offices (doctors, accountants) ,and entertainment. The oldest section of Middletown is the downtown section. This is where the huge library (The Thrall Library) is located. The downtown section also has lots of ethnic restaurants (Mexican and some Indian) as well as bars and night clubs. This area also has some of the oldest businesses in the area, such as classic Italian bakeries and a gigantic owner-operated hardware store. You will find classic brick apartment buildings in this area.
As you move out from the downtown area, you will find new apartment buildings. Many of these will have a pool on site. Some of these buildings are co-ops, but there are others that are designed strictly for renters.
The train station is located near the enormous Crystal Run Galleria. This is the epicenter of commerce in the area. You will find everything from Home Depot and WalMart to small independent liquor stores and pet shops. There are also lots of restaurants along the main strip (route 211). The train station has a huge parking lot, but you could also get a taxi from your apartment in the morning and evening.
Another cool spot to look for an apartment is Port Jervis. This city is located right on the banks of the Delaware River. If you're looking for a scenic area, then you can't beat this spot. The mountains rise up in the distance, and the river is easy to walk to. You can go kayaking on the weekends or hike in the mountains. The city has around 8,000 residents. There are a bunch of shops (pharmacy, pizza place, cafes, hardware) but most people do grocery shopping in the neighboring town of Matamoros (in Pennsylvania) or Montague (New Jersey). Port Jervis is located right at the boarder of the Tri-State, so traveling into New Jersey or Pennsylvania is not an issue, as they are immediate neighbors.
There are lots of apartment buildings located around Main Street and Pike Street. These are both within walking distance to the train station.
You can visit sites like http://www.daleforestapartments.com to look at specific apartment rentals.