Zach,

> My hunch though is if your data warehouse is of a

> limited geography, city or county, wide then

> pre-projecting data is still the way to go. Choosing

> an appropriate state plane would remove the risk

> of inaccurate measurements. Any thoughts?

I agree. In instances where:

1. it’s reasonable to expect that the area of interest won’t exceed the useful bounds of a single, local, projected space, and;

2. the data creators/consumers desire to use the same spatial reference;

then it makes sense to physically store the source data using that spatial reference. As you stated, it removes the potential for risk in repeated projection operations, in the event that we’re wrong about the safety of project on-the-fly. Perhaps more importantly, this approach avoids the need for datum transformations, as Michael Olkin discussed, which we know can be disruptive. Lastly, avoiding the need to project/transform on-the-fly improves performance, but perhaps not to the degree that a city/county scale ArcMap user would notice; still, it doesn’t hurt.

With respect to Michael O. and Tripp’s earlier comments on project vs. project on-the-fly, I agree that not everyone who encounters the ArcMap coordinate system warnings understands the best way to mitigate them for any given set of data sources. I’ve chosen to address this in the past using techniques such as:

· Provide the users with preconfigured ArcMap documents that have the appropriate projections / transformations incorporated into their data frames

· Develop standard operating procedure documentation that describes how to create a map for editing

· Use a versioning model with permissions that enforce the need for a supervisor / QC lead to post to a publicly-visible version (and, presumably, to review the edits before they do)

· Enable geodatabase archiving and DBMS-level protections (e.g. point-in-time recovery style backup for any DBMS; flashback database for Oracle) to allow for diagnosis and recovery of earlier revisions of a data set, if errors are discovered in the distant future

These solutions all presume some sort of structured environment – and, in the latter two cases, an enterprise geodatabase. There’s probably no safeguard against the lone GIS analyst who makes a hasty choice in the interest of expediency; lord knows I’ve flipped a coin on choosing many a datum transformation method in the past. (Emphasis on “past”; for any prospective clients reading this, I don’t do that kind of thing anymore :)

Guiding people towards projecting the source data may incidentally achieve the same goal. But, I don’t know that my confidence in the results would be increased by sending a user who is inclined to ignore the ArcMap project on-the-fly warning to the ostensibly harder Project geoprocessing dialog, instead. We all concur, though, that good training is paramount. The UIs are getting easier and software defaults are getting more intelligent as the years go on, but the fundamental relationship between spheres and planes stays the same.

-Michael

Michael Mannion | Database Consultant : Principal | Mannion Geosystems | +1 617.939.9958 | [log in to unmask]

From: Zachary Fisk [mailto:[log in to unmask]]
Sent: Wednesday, April 30, 2014 8:30 AM
To: Michael Mannion; [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

Great summary Michael Mannion. I

ve been managing a large geo-warehouse of national data since the nineties in coverage format. I've only experimented once with project-on-the fly in Arcplot, my focus is always to project data on the backend for efficiency, we have a common defined projection used to store all our data. Projecting data occurs during our ETL process, since ArcPlot doesn't project-on-the fly unless you specifically invoke it, the tool is wonderful to confirm data alignment. We also perform many spatial operations at the Arc prompt dictating the data to be in a common format.

In the modern world of today's data and where we use newer platforms project-on-the fly is favored since you it's fast and the ability to perform analysis operations are permitted between datasets that have differing, but defined, projections.

My hunch though is if your data warehouse is of a limited geography, city or county, wide then pre-projecting data is still the way to go. Choosing an appropriate state plane would remove the risk of inaccurate measurements. Any thoughts?

Zach-

From: Michael Mannion <[log in to unmask]>
To: [log in to unmask]
Sent: Tuesday, April 29, 2014 9:54 AM
Subject: Re: ArcMap Layers Not Lining Up

Friends,

Michael's comment below touched on one of my perennial GIS holy grails. At the risk of hijacking this thread, I'd like to ask the group for some more feedback on this topic:

>
> If you want to display your Lat/Long data in a Cartesian
> coordinate system, do not rely upon Projection on the Fly.
>

I've set out to prove or disprove this a couple of times, and have found empirically that project on-the-fly is reliable in an ArcGIS Desktop / geodatabase environment. But, I'm not smart enough to prove it either way. If you have time for a page or two of additional reading with your morning coffee, I'd appreciate your feedback on my experiences below.

ARCINFO WORKSTATION

I was taught in the Workstation ArcInfo days that projecting a coverage to a new coverage was better than projecting on-the-fly in ArcPlot/ArcEdit. It might have been true, but I was too busy trying to memorize every command back then to bother testing the axiom first-hand. So, I followed the guidance of those who went before me and did a real PROJECT when I cared about the results.

ARCGIS 8.X

By the time ArcGIS Desktop and the geodatabase arrived, I had the time, interest, and imperative (namely, writing a class) to look into project vs. project on-the-fly more closely. Working with a colleague, we devised a series of tests using Desktop/geodatabase in an effort to observe differences between actual and on-the-fly projections.

In every case, the results from the two projection methods were identical. So, in at least *some* cases, project on-the-fly was safe. But, this didn't prove that both projection approaches were universally identical. From talking with people who knew projections way better than we did, we gained additional confidence that the results should, indeed, be the same whether projecting the source data set or using project on-the-fly in an ArcMap data frame.

Still, no one was willing to go on record as contradicting the conventional wisdom that project on-the-fly was inferior. Facing a deadline, we ultimately caved in and included in our class a statement to the effect of: projecting on-the-fly may be accurate, but for the best results, it's safest to project the source data set.

ARCGIS 9.X

Fast forward to our third and last time period: the ArcGIS 9.x era. I worked on several projects that each required designing geodatabases for geographically large areas of interest. In these cases, we wanted the seemingly-conflicting benefits of:

1. Seamless data across multiple UTM/SPCS/etc. zones

2. Accurate, local projections for editing

3. Coordinate data integrity over time / multiple edit sessions

It seemed like the best option was to store everything in geographic coordinates, then to have each editor project on-the-fly into their preferred local coordinate system during their edit session. But, we were wary of the old adage that project on-the-fly was bad for anything other than eye candy - and, of course, unsure whether or not this was true.

A new series of tests showed project on-the-fly to be accurate and stable. More importantly, talking in detail with Esri staff at the UC one year also confirmed that ArcGIS Desktop used the same algorithm for projecting vector data on-the-fly as it did for projecting source data. So, we took the plunge on a few systems by storing the source data in geographic coordinates and projecting on-the-fly in ArcMap edit sessions. And...

Everything worked. As far as we know.

Could it be that there are some deleterious changes occurring during the edit sessions that none of us observed during testing, and that no one notices today? Perhaps. I think it's equally likely that project on-the-fly is just fine for editing nowadays. But, I don't know enough to prove it one way or the other.

If anyone has evidence either way to support the difference or similarity of projecting source data and projecting on-the-fly, I would be interested in learning more from you. I'm most directly concerned with ArcGIS Desktop/Server, with geodatabases, with vector data. But, if there are other environments that warrant comment, please feel free to do so.

Thanks, in advance, for your feedback.

-Michael

Michael Mannion | Database Consultant : Principal | Mannion Geosystems | +1 617.939.9958 | [log in to unmask]

-----Original Message-----
From: Northeast Arc Users Group [mailto:[log in to unmask]] On Behalf Of Olkin, Michael
Sent: Monday, April 28, 2014 2:47 PM
To: [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

Rule of thumb: If you want to display your Lat/Long data in a Cartesian coordinate system, do not rely upon Projection on the Fly. Re-project the data from Lat/Long to whichever system you want to use & you will usually notice a difference between the input & the output.

Mike Olkin
GIS Administrator, Town of Amherst
4 Boltwood Avenue, Amherst, MA 01002
PH | 413-259-3247 FX | 413-259-2403
www.amherstma.gov/maps | @amherstgis

-----Original Message-----
From: Northeast Arc Users Group [mailto:[log in to unmask]] On Behalf Of Quodomine, Rich (DOT)
Sent: Monday, April 28, 2014 1:31 PM
To: [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

Address locators can be notoriously off between transformations and systems. For example, does the basemap road that was used to make the locator take into account the lane widths, numbers of lane, centerline, etc? Does it take into account unusual numbering systems (For example; Queens, NY)? How good was the base address data to begin with?

What Tripp and Jarlath have suggested are likely solutions, but even with the best of solutions, street addressing is a process that is as much finding (or re-finding, perhaps) a needle in a haystack.

-----Original Message-----
From: Northeast Arc Users Group [mailto:[log in to unmask]] On Behalf Of Tripp Corbin
Sent: Monday, April 28, 2014 1:19 PM
To: [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

Evan,
There are several reasons you might need to spatial adjust data that was created using X,Y coordinates. Here are a couple I can think of:

1. The accuracies of your data and the original X,Y data are different. Even if collected with GPS/GNSS, not all receivers are equal. Not to mention all the different factors such as PDOP, SNR, Multipath, and more that can impact the quality of the data collected.

2. Differing transformations used when projected. If different transformations were used when the data was projected this will impact how well the data overlays one another even when in the same coordinates system.
This most often happens when someone that understands transformations actually take the time to pick the best one and others ignore it using the default or none at all.

3. Different control were used for development of the data. This could be a different basemap or survey control.

Tripp Corbin, MCP, CFM, GISP | Chief Executive Officer eGIS Associates, Inc.
[log in to unmask] | www.egisassociates.com
678-710-9710 ext 21 | 866-304-3864 Fax
Esri Certified Trainer | Esri Certified Desktop Associate | Esri Certified Enterprise System Design Associate

-----Original Message-----
From: Northeast Arc Users Group [mailto:[log in to unmask]] On Behalf Of Aird, Evan
Sent: Monday, April 28, 2014 12:29 PM
To: [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

Thanks to everyone who has responded. Tripp, I followed your workflow and am thinking that it may be necessary to do spatial adjustment, although the data was projected from XY data upon loading it into a file geodatabase so I'm not sure why it needs to be spatially adjusted. In terms of performing a spatial adjustment, I may have to geocode some of the addresses associated with lat/lons in the problem layer and line up the ones that were lat/lons with the corresponding ones that were street addresses. Thoughts?

Evan
________________________________________
From: Northeast Arc Users Group [[log in to unmask]] On Behalf Of Tripp Corbin [[log in to unmask]]
Sent: Monday, April 28, 2014 10:03 AM
To: [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

Jarlath is most likely correct. It seems that many users do not understand how to properly project data from one coordinate system to another. They will use the Define Projection tool or just change the Coordinate system under the Shapefile or Feature Class properties thinking this will project the data from one coordinate system to another. Which as many of us know, those methods do not project the data to a new coordinate system. They only assign a different one which causes headaches.

Here is my method for fixing issues like this. There is no real automated way to do this.

1. Delete the projection and coordinate system for the Feature Class or
Shapefile that is questionable

2. Open a blank map and add the above mentioned data. You will get a
warning about it missing a spatial reference. That is ok. It is what you should see.

3. Look at the coordinates you see as you move your mouse in the map
display. See if they look similar to other values for data you work with that has a properly assigned coordinate system. If they are in the 0,0 or
5000,5000 or 10000,10000 range, then it is possible the data is not georeferenced at all and will need to be. This often happens with data that was created by surveyors or engineers and converted from CAD.

4. If you identify some existing data that you think may match your
questionable data, add it as a layer. If they align, then you have identified the correct coordinate system for your questionable data. If not time to move to trial and error method.

5. With the data you added above that is in an established and verified
coordinate system, start changing the coordinate system to the data frame until you get the questionable data and the known data to line up. I would recommend starting with the following:

a. Local State Plane NAD 83 (original NAD 83 not HARN, CORS 96, 2011)

b. Local State Plane NAD 83 variants also try different Units

c. Local State Plane NAD 27

d. Local UTM Zone

e. Geographic WGS 84 (especially if data was captured with GPS)

f. Web Mercator Auxiliary Sphere

g. Start running through any other coordinate systems commonly used in
your area

6. Once you have identified the coordinate system, use the Define
Projection tool to assign the correct coordinate system to the data in question.

7. If the data is not georeferenced, you can use the Spatial Adjustment
toolbar to move it to the correct location in the coordinate system you desire.

I hope this helps.

Tripp Corbin, MCP, CFM, GISP | Chief Executive Officer eGIS Associates, Inc.<http://www.egisassociates.com/>
[log in to unmask]<mailto:[log in to unmask]> | www.egisassociates.com
678-710-9710 ext 21 | 866-304-3864 Fax
Esri Certified Trainer | Esri Certified Desktop Associate | Esri Certified Enterprise System Design Associate

From: Northeast Arc Users Group [mailto:[log in to unmask]] On Behalf Of Jarlath O'Neil-Dunne
Sent: Monday, April 28, 2014 9:37 AM
To: [log in to unmask]
Subject: Re: ArcMap Layers Not Lining Up

This means that one of the layers has a coordinate system that is defined incorrectly. This video may help.
http://www.uvm.edu/~joneildu/Video/GIS_Open/CoordinateSystems/CoordinateSyst
ems.htm
--
Jarlath O'Neil-Dunne
University of Vermont | Spatial Analysis Laboratory
802.656.3324

[cid:[log in to unmask]]
Aird, Evan<mailto:[log in to unmask]>
Monday, April 28, 2014 9:24 AM
Hi everyone,

I am having trouble with layers that don't line up with each other in ArcMap. When I go to Source in Layer Properties all layers have the same Projected Coordinate System. I found the resource at http://www.gsd.harvard.edu/gis/manual/projections/ informative but did not take away any possible solutions from it. Does anyone have any suggestions?

Evan
-------------------------------------------------------------------------
This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.
-------------------------------------------------------------------------
This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

-------------------------------------------------------------------------
This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

-------------------------------------------------------------------------
This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

------------------------------------------------------------------------- This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

------------------------------------------------------------------------- This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

------------------------------------------------------------------------- This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

------------------------------------------------------------------------- This list (NEARC-L) is an unmoderated discussion list for all NEARC Users.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.

If you no longer wish to receive e-mail from this list, you can remove yourself by going to http://listserv.uconn.edu/nearc-l.html.