120 Hours in the Python’s Den: PyCon 2011

Starting in Version 10, Python is the de facto geoprocessing scripting language for the ESRI suite of software.  I primarily use python to automate the data processing work I perform.  I also use Python to create geoprocessing scripts for the ArcServer environment.  This year I attended PyCon, the Python languages community convention, to further develop my Python skills and see how others are using this powerful language.

I started my excursion to Atlanta for PyCon with three questions in mind:

1)      How can I utilize the multiprocessing capabilities of the python language?  What obstacles have others faced in implementing this functionality with ESRI’s ArcPy package?

2)      I would like to create intuitive input dialogs for my geoprocessing tasks.  How can I leverage other GUI frameworks (Qt, TK, wxPython) to create input dialogs with more built in functionality?

3)      How can we utilize the cloud to reduce the time for complex and time intensive geoprocessing tasks?

I took four classes to start the conference: Python 101, Google App Engine, Creating GUI Applications in Python using Qt I, and Deploying web applications to the cloud.  I learned a few cool tidbits from these four classes.

  • The print statement will be replaced with a print function in Python 3.  Syntactically this change makes a lot of sense.  In case you are wondering, you can actually backport this functionality into the Python 2.6 that comes with Version 10.  Quick code sample:

<code>

# Old

>>> Print “%s is a very fun language for %s” % (‘python’, ‘geoprocessing’)

# New

>>> Print (“{0} is a very fun language for {1}”.format(‘python’,’geoprocessing’))

>>> Print (“{langauge} is a very fun language for {task}”.format(language = ‘python’, task = ’geoprocessing’))

</code>

  • Google App Engine is cool, but I do not have a reason to use it yet.  I feel in the future, using this service, I will be able to quickly stand up a site without having to worry about infrastructure.
  • Qt can run on almost everything and every operating system.  This is a huge plus, however I am going to hold judgment until the dust settles with Nokia.
  • In a little under three hours I was able to setup a distributed, load balanced django application on the Amazon Cloud pulling data from their Relational Database Server and distributed using their CloudFront CDN.  I definitely see this as a resource I will invest more time in learning.

It is amazing to see what people are doing with python, primarily in the distributed computing and web application end of the python development spectrum.  My new goal for the year is to finally get around to learning to develop with Django.  It seems every other presentation was a huge django proponent.  There were also a few interesting talks on the forecast of Python language development and IDE’s for writing your code. Fortunately, for you the Python Foundation videotaped most of the presentations this year, http://pycon.blip.tv/posts?view=archive.

Alright and onto a few takeaway lessons I learned from my experience at PyCon.

  • ESRI is investing tons of resources into building Python as a quick, efficient geoprocessing language for their tools.  PyCon ran concurrently with ESRI’s developer summit; however, they were still able to send out 7+ representatives to PyCon.  I spent a couple nights with the ESRI folks, and to be honest I am incredibly pumped about version 10.1.  Tons of cool new features are coming down the pike.
  • Microsoft is gradually pushing into the Python/Open Source community.  I am in the process of downloading Microsoft’s Python extension for Visual Studio.  The latest and greatest release promises to debug both cPython (what ESRI uses) and IronPython (a .Net variety).  Currently I am using WinPython, the default with ArcGIS, but I see the Visual Studio version quickly superseding its use.
  • The python community could learn a lot from the GIS side of the house.  Almost every presentation involved crunching data, processing it in some way and displaying it graphically.  I definitely see an opportunity to step in and show a more intuitive method for displaying spatial data.

Augmented Reality using ArcGIS and Layar

Check out Patrick Barnhart’s presentation from the Developer Summit on ArcGIS and Layar: http://resources.arcgis.com/gallery/video/arcgis-for-ios/details?entryID=CA9F541A-1422-2418-88FA-E7DAC681E831

Esri Developer Summit: Closing Thoughts

Dan Levine

Well the 2011 Dev Summit wrapped up with a bunch of folks walking around with sore arms from the dodgeball tournament and you could tell Jim McKinney was disappointed his team wasn’t  in the finals. As has been the pattern over the last 6 years, Esri raises the bar a little bit each year making this better and better. The sessions we better, they were nimble in moving rooms around to accommodate crowds, and the food was better.  This is by far and away the best technical show they offer, and with the direct access to the product teams, it really is a must attend.

One of the big observations I have from the show overall is that we as an industry are still trying to figure out the mobile market and how to provide Enterprise solutions in that space. What I saw was a lot of sessions on “how to” with iOS or Android or Windows Mobile, which were well attended and necessary. But I am starting to suspect that the attendance in each of these was largely along personal favorites (iOS, Android, Windows Mobile) and perhaps not so much on need.  I was surprised at the low attendance at the couple of sessions about how to pick which one is appropriate or if you want to do multiple platform deployments, how do you decide.  Esri actually did a real nice job in these sessions.

Trying to put that observation or perception in context of the Keynote speakers point that the mobile market is incredibly fragmented right now, and the end users are demanding solutions working on multiple platforms is where I am struggling.  Are we as technologists still so caught up in the coolness factors of each of the individual platforms and the Marketing hype that drives separation (mine is better than yours) that we aren’t seeing the bigger picture? Or is it that we are still in the early part of the technology curve where we have to understand each of the technologies first then figure out how to tie them together?  That is not to say that there aren’t tools that allow us to start doing just that; it just seems like these are just getting out of the gate.

During the week before the Dev Summit I must have received 2 emails a day from some vendor, integrator, or techno think tank group offering to help me as a CTO figure out what my corporate mobile strategy was going to be.  This is obviously a big emerging problem for corporate IT departments that we need to be smart about. Do they standardize on a single platform (a strategy our friend from Gartner would seem to indicate is doomed), do they allow, and therefore have to support, multiple platforms across their IT infrastructure, or do they create some sort of Enterprise Mobile Service BUS that abstracts the layer between core IT systems and whatever mobile platform you want?

Hmmm, that last one sounds a lot like the AGS Server architecture model, and by the way the same model that the run time solution is going to follow……..

Okay slight change of topic, but still related.

So I have heard a little bit about the QT development environment in the last month or two. It’s always around developing cross platform solutions for mobile phones. It has just started to show up as one of the last bullets on Esri slides about developing multi-platform solutions in mobile. But for the first time I see that Esri has fully embraced this as part of a product line in the development of SDKs for the run time solution they will release as part of 10.1. On digging into the resource center, I am seeing a modest amount of whitepapers- like instructions and getting started material for using Qt for engine development- in 10.x. Man how do you keep up with all of it!

Qt is a cross-platform application and UI framework. Using Qt and C++, you can write web-enabled applications once and deploy them across desktop, mobile and embedded operating systems without rewriting the source code. http://qt.nokia.com/.  The guys presenting the run-time special session mentioned that it has been used for large scale enterprise solutions particularly when high performance is required.  Some of the real-time map refresh capability they showed during the demonstrations apparently will be available first with the Qt SDK and follow with the other 2 SDKs (JAVA and Silverlight WPF).  Patrick sat in a session for developing military solutions and they showed a Qt based application with military operations, moving maps, real-time comms, etc.  I was also a bit surprised at the show of hands during the run-time talk, apparently there are a bunch of C++ developers in attendance. I don’t remember seeing that represented much previously.

Well over the next couple of weeks I will be doing a lot of thinking about what we saw this week and trying to figure out what it means to the industry and what we as a company want to invest in. Already looking forward to next year and what we can do until then.

Steve Mulberry

The Dev Summit came to a close today but not without its share of informative sessions. I sat through one this morning that made me think more about the comment, “ArcGIS Desktop is not meant for the web”.  We keep hearing how GIS is moving to the web with the many open API’s, free templates, tools for hosting and managing data,  applications, etc. I also hear Esri and others say, “Desktop will always be around and there’s no way you can do the things in desktop over the web”. Not so fast; in the last year or so we’ve seen BA Desktop become BA Server with an open API, and then evolve into BA Online.

This opened the door for Community Analyst.

Early this year we saw ArcLogistics desktop become web enabled.

Now I learn that the desktop redistricting tool we’re all familiar with has moved to the web.

Each one of these tools is arguably a full featured GIS with editing data, analyzing information, map production, and dissemination. What’s next, an online property management system? Desktop over the web…not such a pipe dream any more.

Patrick Barnhart

I’m sitting in the airport on my way home from the summit and it seems a good time to wrap up.  It’s been a great week but it’ll be nice to get home.  I’ve taken in so much stuff that it’s starting to blur together but since this will be my last post for a while, I thought I’d focus on a task that’s common to a lot of the projects going on at GISi – printing high quality maps from a web application.  I mentioned this one on day 3 and promised to attend some sessions on Python to learn a little more about it since that seems to be the easiest way to go about printing to a PDF from ArcGIS 10.

Python has been part of ArcGIS since release 9 and has been successful enough that support for it has been increasing with every release until now, at 10.1, it is fully integrated into the application – intellisense, code completion, and integrated help; they’ve even added a Python console to the main interface and gave it a name – ArcPy.  I’ve messed with Python before, mainly by creating tools using Model Builder and exporting them to scripts that I could modify, but I’ve always used it inside of ArcMap, which isn’t very helpful when it comes to printing from the web.  One of yesterday’s sessions entitled ‘Using Geoprocessing Services in a Web Application’ showed me that there’s a lot more that can be done with it, and the follow up sessions I attended on Python make it look pretty easy.  Python is a very clean, easy to read language, and while I’ll have to learn a new syntax, it looks fun.

Okay, so we all know you can publish a Geoprocessing task through ArcGIS Server and make it available to other ArcMap users via the toolbox.  What I didn’t know, or at least hadn’t really focused on before, is that you can also access these tools via REST.  As we’ve seen in my previous posts, there’s a number of ways to make REST call to ArcGIS Server, either directly using a URL from a back end web service or from the front end using one of the Web APIs (JavaScript, Flex, Silverlight).  Any of these will work; I haven’t done much Flex or Silverlight, but me and JavaScript go way back, so that’s the approach that appeals to me.

As we should with all projects, let’s first define our scope.  What I’m after is the ability to allow the user to select a template from a list; this will set the locations of the map elements on the page, as well as the size and orientation.  So the title, legend, scale bar, north arrow, comments, etc will be placed where we want them to go so we can make a nice clean map that’s easy to customize if we need to adjust the template.  Then all we need to do is create the elements, put them in where they need to go, and output the whole thing into a PDF for the user to print at a high resolution.  Piece of cake, right?  If you’re doing it from scratch, maybe not so much.

ArcPy, however, makes things easier.  There are a ton of existing scripts that we can use and it has the ability to open an .mxd to access the layout and retrieve map elements based on their name.  For example, a text area named Title can be retrieved and populated by ArcPy with relative ease.  To do this, all I really need is to create layouts in ArcMap (one layout per .mxd, named appropriately so I can find them later) and make a JavaScript front end with a couple of menu items and a print button.  Esri is all about Dojo (some folks like JQuery, I’m more of an ExtJS guy), but it doesn’t matter as long as we use the JavaScript API to create a handler on the print button, we’re good.

Our handler needs to get the state of our map (extents, layers, etc) and the layout selected by the user, then make a REST call to a Python Geoprocessing tool that will do the work.  Of course, I haven’t actually done this part yet, but since we’re dealing with ArcGIS templates, there’s existing code we can use to generate the map and export the whole thing to a pdf.  The example I saw, which should be available on ESRI’s resource page in a couple of weeks if it isn’t up already, was only a couple of pages worth of code, and compared to how I’ve done this in the past using open source libraries like ITextSharp, it looked really simple.

So, as you can see, my head is still spinning from all the possibilities I’ve seen over the last couple of days, and want to play with all the new toys all at once, but the reality is there’s only so many hours in the day.  Hopefully I’ll get a chance to take a closer look at some of these things in the days to come and will pass along what I learn.  And finally, if you’re like me and have never had a chance to attend the developer summit before, start bugging your manager about it for next year; it’s a good time.  Traveling is a pain and there’s a lot of sitting still involved (which is tough for me), but you’ll see a lot of great stuff and Esri puts on a great show.

Ben Taylor

I attended a session today geared toward the defense and intel community. The majority of the presentation centered around creating maps for military use and 2525C symbolization, however, there was a cool demo that utilized the upcoming 10.1 runtimes, specifically the C++ Qt SDK. The demo included messaging between multiple simulated vehicles. The performance of the application running on the new runtime was quite impressive.

And speaking of the new runtime, I also attended a session highlighting their upcoming release in 10.1. There are some pretty significant changes coming that will yield a smaller overall footprint, increase performance, and simplify deployment. However, low level access to ArcObjects will not be supported in the runtime. So there will be a shift in development paradigms mainly toward WPF. A word of advice I picked up in the presentation is to get familiar with the Silverlight Web API, because if you are comfortable with that, you’ll have a good head start on working with the new runtimes.

Tommy Bramble

I started the final day of the conference with the ‘Building RIA Clients Using ArcGIS Workflow Manager REST Services.’  Esri is calling AGS Server Workflow Manager the ‘complete GIS management system’ and recommends using the product to create and manage GIS project workflows.  The product can be used by all stake holders including clients, managers, analysts, and developers.   Workflow Manager will allow you to author and publish project workflows, centralize project information and documentation, control access to data (including spatial data in your SDE repository), and integrate GIS into business processes.  The main intent of this session was to demonstrate how you can build a custom workflow manager solution using the new REST endpoints available.  The presenters showed examples of a custom Flex viewer that was able to query users, jobs, reports, and workflows through attribute criteria or by spatial area-of-interest.

My next session of the day was titled ‘Redesigning Desktop Applications for the Web.’  The session was basically a talk on why you would (and wouldn’t) want to move your applications to the web.  The web has become a pervasive, accessible, and proven platform and now might be the best time to consider moving desktop apps over, but, ultimately, you will need good project planning and a sound understanding of the user audience to make the best decision.   Some of the key information I took away from this session included:

  • The web is a fast and cheap infrastructure that provides a platform for easy collaboration and sharing.
  • Application development is easier now than ever before (with WebAPI’s and Application Builder tools).
  • The web interfaces are richer and smoother, allowing a desktop-like experience.
  • The user experience on the web is highly important.  You need a simple, but powerful application.
  • You must know the user audience and application requirements.  In some situations, a desktop application will still be the best choice.
  • Must consider security implications on the web.
  • Planning is key!  Planning the design, development, and deployment phases are critical to any application’s success.

My last session of the conference was ‘Advanced Development with ArcGIS API for MS Silverlight/WPF’.   Having no development experience with Silverlight, my goal was to learn what is possible with the framework and how it may differ from the other WebAPI’s.  The presenters here, like in most all other web sessions, stressed the importance of the user experience and reminded the audience that the application does not always have to be map-centric.  The application examples presented here demonstrated the grid/map interaction (i.e. hover over a data record and an assortment of animated effects highlight the related map feature) and other visually rich ways of interacting with the map and attribute data.  Some basic editing examples were also shown.  From the standpoint of a beginner Silverlight developer, I was most interested in the new Silverlight Application Builder (a WYSIWYG Silverlight editor) and the Visual Studio Silverlight templates.   Apparently, you can easily switch back and forth between the Application Builder and your custom Visual Studio-based Silverlight projects, which could make development in teams easier.

This was the end of the conference for me.  Esri did a great job and presented a lot of useful information regarding their products and future trends.

Esri Developer Summit: Day 4

Danny Bradshaw

Alas, all good things must come to an end… but not without a few more great technical sessions and some parting words from Esri.

The morning kicked off with Advanced development with Android.  While this session was heavier in code than the Intro to Android session I attended earlier in the week, the takeaway for me was the reaffirmation that this new API will be a major player in Esri’s API “toolkit.”  The development team demonstrated building an application from scratch to completion, but made a few quips I wanted to share… 1) The Android API team is largest platform specific API team employed at Esri, 2) The Android team claims they will be the first to enable disconnected mobile editing using a targeted platform API (obviously disconnected editing is already available with AGS Mobile).  Esri is committed to Android, and their excitement made me want to build and deploy a native Android App.

Next was a discussion concerning the Yetti of the Esri API stack… the File Geodatabase API.  The demo was dry and monotone, which combined with C++ programming created a slight glaze over my ability to focus.  However, what kind of a developer summit proxy would I be without a few bullet points of info:

  • Only available for 10.0+ File Geodatabases (due to metadata structure change with 10.0)
  • Can create, open, delete, read, edit,  etc. on all simple features; Can read complex features
  • Can perform attribute queries, and very limited spatial queries
  • Supports a subset of SQL92 standard SQL queries
  • No raster or attachment support (yet)
  • They’re working on creating DotNet and Java wrappers ASAP

The technical sessions ended with an inspiring and animated Flex demonstration with Mansour Raad.  As a Flex developer, it’s unfortunate that this was the only Flex session I was able to attend, however, this was my best and last session of the summit.  The team announced the FeatureLayerTask (an object allowing editing that doesn’t require a feature layer).  Mansour was also kind enough to treat this session as an advanced session (sometimes advanced sessions discuss how to add dots to a map) and covered Flex heat maps (using bitmap blitting – http://en.wikipedia.org/wiki/Bit_blit), custom clustering symbols, and discussed best practices for flex software development.  Mansour’s discussion reaffirmed to me that decisions we made with some of our recent development projects were good ones.

Esri Developer Summit: Day 3

Dan Levine

So Esri kicked off Wednesday announcing the winners of the Developer Challenges.  Frank Roberts won the Developer Challenge with what looks to be something like the Tri-View Tool that we have been deploying for a couple of years. Not sure about the details, but it would be worth looking at to see if we can leverage each other’s work. Some of you may know Frank; we have interviewed him a couple of times trying to figure out how to make a fit within GISi. Congratulations to Frank.

Next was the Keynote. Apparently a last minute substitution by the Gartner Group sent a solid number 2. Michael King, Research Director, Gartner, spoke on The Future direction of Mobile Application Development.  He had many different stats and charts talking about how the mobile industry is incredibly dynamic now and how that impacts us as developers and solution providers. A couple of the interesting stats:

  • In the next few years 70% of all interaction will be through our cell phones
  • Mobile is a Trillion dollar business.
  • Industry is VERY fragmented and don’t see a consolidation for a least of couple of years if it ever happens, largely driven by how personal the use and experience of mobile devices are.

Above and beyond the statistics and observations, Michael presented that he believes that the successful solutions will be the ones that most effectively harness the ability to be context aware and have the most elegant interfaces. Specifically around context aware, not only is he referring to where you are (GPS), what you are doing (calendar), what you are supposed to be doing (reminding you that your anniversary is tomorrow while you drive by the florist that you bought flowers from last year), but tying this all to business and personal data systems to the point where the mobile device is keeping track of more information about you than you possibly can remember at any one time. Couple that with an interface that adjusts to how you use the phone and learns what your preferences are over time.  For instance, for me that might mean that when I travel I always look for a CrossFit gym local to where I am staying. Well instead of me going to Google on my phone and stumbling through entering in “CrossFIt XXXXXX” for wherever I am, it would recognize that I am about to travel and just performs the search and gives me a list of results in the format that I typically look for: contact information, directions, and schedule. Within this theme of contextual awareness he talked a little about Augmented Reality and how the industry is still trying to figure out what we can do with this, how we implement it, and that we have to wait a bit for the hardware technology to progress to make this really work well. An example he gave was a crew in a manhole looking at a 100 year old valve. The only guy in the city that knows how to maintain that valve is back in the office but can share the camera screen from the guy in the field and do live annotation to walk the crew through what to do and not to do.  This did tee up the presentation that Pat did later in the day well, and packed the room.

I am still somewhat surprised that the Mobile sessions, and there are a bunch of them, aren’t the ones that are packed. Seems like the server sessions are still the most popular.

Steve Mulberry

Check out the recorded plenary sessions here: http://www.esri.com/events/devsummit/videos-and-photos/index.html

Great keynote speaker form Gartner; He discussed the many facets of mobile computing and the trends and predictions for the future. One thing that stood out was the amount of mobile devices projected to be used in the future will by far outnumber the laptops. So planning accordingly with respects to UI/Application development is a key to succeeding in this market space. The speaker noted that mobile development should not be a sideline event but imbedded in application development strategies and procedures. We keep hearing the same ole adage about making it simple and easy to use. But we GIS geeks need to think outside the laptop when it comes to mobile development. That’s where you the designers come in.

I sat through several sessions dealing with performance, scalability testing and architecture scaling. The common theme always lead back to making sure your map document is optimized and ready for web deployment. Here are some tools that help this process:

  • PerfHeat – http://arcscripts.esri.com/details.asp?dbid=16880. This tool produces a grid showing color coded rendering time ranges across the map extent. Display rendering times are generated for each map scale, presenting a spatial overview of display processing loads.

All these tools along with additional capacity planning and performance benchmarking can be found in the report. http://resources.arcgis.com/gallery/file/enterprise-gis/details?entryID=6367F821-1422-2418-886F-FCC43C8C8E22

Esri unveiled a new GIS Testing Tool that incorporates the above tools and methodologies in an all-in-one tool. Currently this tool is only available through professional services, but it was suggested that once final it may be made available for download.

Patrick Barnhart

The two biggest things that happened today, at least from my point of view, were successfully surviving my presentation on Augmented Reality and having the GISi dodgeball team (named Gone in 30 Seconds in honor of last year) exceed expectations.  The presentation went well; if you’d like to see the on-screen part of it, check out http://prezi.com/iy3lglcdn6xi/augmented-reality-using-arcgis-and-layar/.  Prezi saved the day on this one; it made things a lot more fun to prepare and present than if I had to do it using standard slides.  Gone in 30 Seconds got a bye in the first round, emerged triumphant in the second and then were defeated by the team that went on to win.

As for the stuff I’m really here for, which is to learn as much about developing web applications using Esri products as possible, it was also a success.  I attended a session entitled ‘Advanced Development with ArcGIS API for JavaScript’ which really should have been called ‘Intro to Mobile Development using the JavaScript API plus some HTML 5 and a touch of CSS3’.  So, it wasn’t what I was expecting but it still turned out to be the most interesting session I attended today.

There are APIs other than JavaScript that are designed for mobile development but they’re device specific, which is kind of a turn off for me.  If I’m putting in the time to write a phone app, I want it to run on iPhones, iPads, Androids, Blackberrys, you name it, and if it’ll run on the desktop too then glory hallelujah.  Write once run anywhere is what I’m after and while it seems that the world is not 100% ready for that ,the JavaScript API gets you closer than anything else.  Other than straight html of course, but that’s just too limiting.  The morning’s keynote session talked about the “do’s and don’ts of mobile development, and if you’re just using html, you’ll end up with a bunch of don’ts (clunky interfaces that are a pain to use).

To make a really sweet phone app with a native look and feel without redoing the code using a separate API for each device, you need to use a framework for the front end.  Something like JQuery Touch, DojoX Mobile or Sencha Touch provides the user with nice big buttons and the fat-finger pans and zooms that everyone is used to.  I do a lot of ExtJS development so it would have been nice to see Sencha in action, but JQuery Touch is cool too and that’s what most of the demos were.  These apps won’t work for the desktop but that was pie-in-the-sky anyway; the platforms are just too different to use the same interface.

  • iOS (iPhones and iPads) – this was described as the best browser of the mobile devices with support for SVG overlays, geolocation, html 5, creating and editing vector graphics, etc.
  • Blackberry Torch – this one’s a little more limited in that zooming in/out using a pinch, isn’t supported and it’s a little slower, but it does handle most of the other capabilities.
  • Android – the Droid phones are more limited in certain things, especially since it doesn’t support SVG overlays.  Vector graphics are rendered as part of the canvas so you can’t select an individual feature and manipulate it, although you can create features and add them.  You also have to set a meta tag to disable gestures on the entire page so you can zoom in/out on a map object, but not the overall view.

So, back to what I want, which is the ability to run the same app on as many devices as possible; the way to go about it seems to be using something like Sencha Touch to build the interface and then hook the events and listeners to JavaScript API, which calls to ArcGIS server.  Design the app for the most limited device, which surprisingly enough seems to be the Android, and go to town.

The JavaScript API works the same way as the rest of the Web API family – create a map object, create a layer object, create a feature object, add the feature to the layer, the layer to the map, etc.  There are a lot of built-ins (way more than OpenLayers), but it’s pretty straight forward stuff once you get the hang of it, so I’m not going into it much here.  But now at least I have a plan for building flexible mobile apps and know what tools I’d like to use when I get the time to make it happen.

The session also went into some pretty cool examples of using HTML 5 and CSS3, but I’m not getting caught up in that until it’s actually here.  Quick note: there’s a site called CanIUse.com that tells you which devices support the really cutting edge stuff.

And there was a bonus!  At the very end of the presentation they mentioned something called the Print REST API, which apparently takes the pain out of printing maps with vector overlays, external wms layers, text, etc.  Exactly the stuff I’ve been doing in the open source world for the last couple of months.  It doesn’t help with my current project, but I’m sure I’ll get a shot at it sooner or later.

In a separate session on Python, I also got a glimpse of using ArcPy to support template driven printing to PDFs, which is another of my current tasks (also using open source technology).  So again, I can’t really use it yet, but I foresee great possibilities and hope to learn more about Python tomorrow.

Danny Bradshaw

Day 3 kicked off with an intriguing discussion with Michael King from Gartner Research concerning the pervasiveness and trends in mobile computing over the next few years.  Although nobody has the magic 8-ball (especially in the ebb and flow of the mobile tech industry), he threw out some interesting stats:

  • 214,000,000 tablet devices in the next few years
  • 80% market penetration for smart phones in developed countries in the next few years
  • 2013 – Mobile phones will overtake PCs for internet access.
  • 2014 – 30% of users will use augmented reality with their mobile experience

Some other interesting sessions came from the Esri Prototype Labs and Esri Production Teams (i.e.  those responsible for making sure they hit the deadlines).  While I could write a book on cool stuff, I was particularly interested in an unconventional dynamic physics symbol, in which the bigger symbols would “bully” the smaller symbols and take their place on the map.  The prototype team also demonstrated an interesting augmented reality app that used a sheet of paper with a specific “marker” that was replaced with an interactive map while video conferencing… think green screen.

The development teams/managers talked about the Esri internal development process.  I was interested to see their strategies, which closely follows the Agile (Scrum) development process.  ESRI loves Scrum and was evangelizing that development process.  I’ll be honest, I was bored to tears as the discussion turned to their extensive… extensive QA processes, but be happy to know they’re thorough.  Some interesting stats out of that session:

  • 200 unique developers checking in code daily
  • 95,000 source code files for ArcGIS Desktop, which takes 5 hours to compile (daily)
  • AGS Desktop 9.0 = 6.9 million lines of code;  AGS Desktop 10.0 = 15.97 million lines of code

Melinda Frost

Since I am all about mobile I am going to write this up on my iPhone! Of course it will be shorter because I did have to go to walk-in clinic for a sinus infection. Everyone here is super nice.

Anyway- back to business. Tuesday night at the meet the teams night, I ran into two of the developers for the 10.1 mobile SDK. So, of course, I drilled them about the problem I am having with the WPF map control ‘flickering’. People want it to behave like TomTom or Garmin.  They told me that the 10.1 SDK for mobile will be significantly different. All the API will be pulling from a new core SDK. And they claim it will be WAY better than TomTom in that regard.  I said ‘Great but I need that NOW’. I did leave my contact info with them so I can be in on the holistic testing for this. Wednesday I ‘tackled’ Martin and Jay- the head guys for the mobile SDK dev team. Again same answer.  After I show them the best work-around I have come up with and I start to see some hope (breaking down) from them. They plan to take my code back and look into if that is something they can do sooner. Fingers crossed!

Other conference related info picked up is that if you have developed for Silverlight web app and/or WPF apps, then you are all ready to build a Windows 7 phone app. Same xaml and C# or VB.NET in back end.  Good news for all you Silverlight and WPF developers.

Tommy Bramble

Day 3 started with the conference Keynote Speaker, Michael King, from Gartner Research.  He talked about the past and future trends in mobile technology and how our lives will be impacted as the technology evolves.  Context-aware computing will become the norm where our smart devices, and the services that power them, will know more about us then we do our self.  The smart devices will become even more powerful and ‘aware’ through sensor driven data capture like facial recognition, better speech recognition, enhanced audio and video sensors, and other advanced sensors that require less direct user input to interact with the device.  Mr. King also covered some do’s and do not’s of mobile application development and commented on how augmented reality will become a much more powerful tool as it evolves.  We can expect to start seeing more ‘real world’ applications where augmented reality apps will assist people with work and other things in life instead of just being the ‘cool’ toy.

My first session of the day was ‘Advanced Development with the ArcGIS JavaScript API’.  This session was almost entirely focused on mobile development with the JS API, although the session was not listed as mobile.  The session probably would have attracted a larger audience had they actually listed the course under mobile.  The presenters demonstrated building mobile friendly web apps using jQuery Mobile, Dojox.Mobile, and Sencha Mobile; all using the JS API for the mapping functionality.  The presenters also briefly covered the HTML5 components that are available in most browsers that can also be leveraged in mobile web apps.  In my opinion, there wasn’t much ‘advanced’ knowledge regarding the mobile development.  The demos were very basic examples of wiring up a mobile web app.  Maybe just the fact that it was ‘mobile’ was advanced.  I did have to leave the session 10 minutes early to prepare for my Tech Support session with the Esri techs, and I was told the presenters did demonstrate a really cool drag-and-drop feature of the web app, but you’ll have to hear about that from Ben who was available for the entire session.

Next up was the ‘Building Secure Applications’ session.  This session focused on securing your ArcGIS Server services and the web applications that consume them.  The presenters first reviewed the correct procedures for securing the AGS services using users and roles, and then reviewed how to use the admin security tokens to get access to the secure services.  The session also covered using proxies with your secure services and then taking the security to the next level by securing your application front-end (client).  The out-of-the-box ASP.NET membership provider was used as the examples for creating the AGS server user and roles and for the application security, although a custom membership provider can be used in its place.

I sat in on the ‘Intro to ArcGIS API for Windows Phone’ next.  This topic area was completely new to me, but I was able to walk away for this session with the basic principles of the Windows Phone development.  A point the presenters emphasized repeatedly was that developing for the windows phone was no different than developing Silverlight applications.  Most all the same tools and class libraries for building Silverlight apps are used for Windows Phone, with the exception of a few class libraries specific to the Windows Phone environment.  The presenters also demoed some of their existing Windows Phone applications, like ArcGIS for Windows Phone and the AtoB Navigation application.  The presenters also gave away a Windows Phone 7 at the end of the session to the person in the audience who could stump the crowd with a Silverlight/Windows Phone development question.

Next up was the most anticipated part of the day, the Dev Summit Party.  Esri had a nice setup of games and entertainment as usual with the highlight of the event being the Dodgeball Tournament.  GISi formed the team ‘Gone in 30 Seconds’.  Luckily, we did not live up to that name and actually won our first match-up in the second round (after a first round bye), but sadly were knocked out of the tournament in the third round.  I also took a shot at table tennis, but finished the night with a disappointing record of 1-4 in my match-ups.  After my last competitor destroyed me by a score of 21-8, with a few of my coworkers watching, I decided to retire from table tennis and call it a night.  All and all another great day at the conference.

Ben Taylor

I sat in on another Javascript API session. There is some pretty slick drag-and-drop functionality coming down the pipeline. The presenter showcased a file api by dragging an image directly from his file system and dropping it onto the map within the browser.  I thought that was pretty cool until he dragged a csv file to his browser and the points along with the associated attribute data within the csv file appeared on his map. He also was able to add additional services by just dragging in the link of a rest endpoint from another browser and dropping it on the map. The only caveat at this point is they’ve only gotten the functionality working in FireFox and Chrome. They’re still working on getting this viable in IE9.

In addition they demoed a print tool for Javascript API that will probably be included in the next release. This tool is meant to be used until out-of-the-box exporting is made available in 10.1.

I also attended an advanced Flex session. This being my first dev summit, it turned out to be quite a treat to hear Mansour Raad present. He gave a lively presentation on how to best utilize the MVC pattern to develop Flex applications, so that they can easily be adjusted to fit web and mobile platforms.

GISi Dodgeball: “Gone in 30 Seconds”

TEAM “GONE IN 30 SECONDS” Represents

With a bye in the first round and a solid win in the second the GISi Dodgeball Team made it to the final eight! You should have seen Melinda dance!

 

 

 

 

 

 

 

 

 

The rules changed a little this year. Each match was best of three until the final four then it went to best of five. Even though each game was capped to 3 minutes it is surprising how tired you get.  Everyone on our team was instrumental in our success. Melinda, as the token female, drew fire from the opponents while Tommy was picking them off with surprising accuracy. Steve made an amazing catch with a lightning bolt right to his chest. I still don’t know how he held on to that. Danny’s southpaw also caught the other teams off guard several times. Pat’s stealth kept him in the matches, and Ben made a clutch catch to close out our first match, the look on his face told me that he was as surprised as anyone that he caught it.

Esri Developer Summit: Day 2

Danny Bradshaw

Day 2, like most ESRI conferences, was the official start of the conference and included the customary plenary session.  Some of the noteworthy points:

  • ArcExplorer Online being redesigned to get rid of the Microsoft-esque ribbon (and taking design cues from Apple, I’m still laughing at that one!)
  • Ability to author and publish ArcGIS Online from ArcGIS Desktop
  • ArcGIS Online portals coming soon so you can host and publish your own virtual AGS Online within your own firewall.
  • Application builders for both the Flex and Silverlight viewers
  • Better support for popups to include richer and more relevant information

These points are all well and good, but the really cool demo of the morning came when the team used a Microsoft Kinect camera to zoom, pan, and redline a simple Silverlight map.  Talk about a new take on the touted “smart tables” we’ve heard so much of over the last few years.  Who needs a table!  The demonstrator had to sync with the Kinect system using a “hands up” position (picture someone sticking a gun in your back and demanding your wallet), but after that, it was literally a GIS symphony as he waved his arms around to navigate and edit.

Then Ismael Chivite got on his soap-box for a few minutes.  I found myself chuckling when he declared that GIS professionals make ugly applications and that it’s time to get designers involved (agreed!).  He then talked about how the “black box” of ArcGIS Server is improving.  As a company, we’ve already heard the news about the changes being made to the data and architecture upcoming with the 10.1 release of ArcGIS Server… Ismael touted benchmarks such as the Identify functionality being 200% faster, dynamic map services currently processing 30-40% faster, 60 ms queries, etc.  Pretty impressive stuff on the surface.

As a final note, I attended both the AGS for iOS and AGS for Android sessions.  AGS for Android is in public beta (or perhaps just launched with final), but I have to say, between the two development platforms, I would choose Android in a heartbeat.  The differences between developing with a full-featured developing language like Java vs. what I saw with Objective-C was night and day (sorry if I offended any C aficionados – and forgive my ignorance).  While, in the long run, iOS apps might produce some smooth and sexy apps… You’ve got just that, a long run, because you’ll have to write twice as many lines of code than an identical app on the Android platform.  Of course, I did meet a fellow  who writes iOS apps with C# (MonoTouch)…

Dan Levine

Code for America Introduction

Code for American is a new nonprofit organization built to serve the needs of local governments across the country. The driver is that the economic downturn has impact local governments to provide services to their residents. Take this in context with the large IT aware growing youth community, and the re-developing attitude that we won’t wait for our government to solve our problems. The concept for Code for America is to bring in a set of young technicians (developers, designers, writers, etc.) looking to give back to the community and send them out to a set of cities with need. They interview city stakeholders, community organizations, and the public and identify and build applications to fit an unmet need with technology. Typically these are giving the public a way to find or report on information already in multiple disparate databases in a city.  Furthermore, these apps are designed to be portable to other cities (to the extent possible). One examples given was an application to connected a school bus tracking system to a mobile reporting application so parents could find out where the kids buses were during snow storms. Another was a tool that let the public adopt a fire hydrant; they were then responsible for shoveling out the snow so the fire department wouldn’t have to in the case of emergency. The web application let the adoptees report on status after snow storms and the fire department always had real time awareness of availability. This program is just getting started with the first set of 20 technologists but is already proving successful. Esri is very interested in integrating these efforts into is Local Government Templates model for “canned” solutions.  So turn this around from a business perspective and what does this mean to us. I can see a couple of perspectives. First, this group is out there building solutions for free and giving them away to our customer base. Hmmm, does that shrink our market? Or is this making more and more people aware of IT and GIS solutions and effectively help our cause. On the other hand, we are all interested in more effective government and public involvement and this seems to be enabling this well.

Manseur Raad, Kinect in Flex

Well Manseur must have got his feathers ruffled during the plenary when Art Hadaad showed a Silverlight app and Kinect because this talk was supposed to be about Augmented Reality. Instead Manseur decided to show the flex version of what Art showed. Esri technology competition? You tell me.  Any Manseur walked through the available tools and how to implement a solution that connects flex based map applications with Kinect. Well he didn’t disappoint. Using Adobe Air and Open Source connectors from TUOI.org he walked through the code and demonstrated how to build a simple application controllable through the Kinect sensor.  Really cool stuff and my mind is full of ideas of how this might be useful.  Sporting his Flash Gordon T-shirt Manseur had the crowd on the edge of their seats even getting collective Amens from the audience – a strange and funny cultural event. Manseur will be making the code available, if you follow his blog you will see it shortly. http://thunderheadxpler.blogspot.com/

Choosing a Mobile Deployment Platform

My first take away from this session was that I was stunned at how few people were actually in there.  This was held in the largest room but was attended by only about 50 people. Is no one else getting the push to develop Mobile applications? Perhaps not or maybe this is still such a developing technology that people are still shying away.  Anyway this session was all about pros and cons of deploying a web application that works on any mobile device developing and deploying on a device specific applications.  Some of the pros/cons for a web application are that it is mostly a single development and deployment with some modification for each platform, you have total control over deploying and updating the application, limited access to device capabilities (camera, compass, gps, etc). While the device specific solution offers better performance and function, usability – designed specifically for the device, direct access to device capabilities, market penetration – app stores, easier to monetize. The downside is that you have to build multiple solutions for each platform – if you need that, which drives up development and maintenance cost, you typically have to go through some gatekeeper to make your app available.  They did show the Adobe Air option where you can build in one environment and deploy on and Android and an Apple device.  They also talked about a couple of other emerging technologies. JavaScript compact build and JQuery mobile and PhoneGap were two prominent ones.  Mentioned the tool Protofluid, which works with Firebug to test and troubleshoot phone behaviors on your desktop. The remainder of the session was about each of the mobile device specific development platforms and the pros and cons of those and they took an informal survey on who was going to deploy on which of these – basically asked us to scream for each one and they measured noise. Results http://stlouis.esri.com/apps/dsmobile. Not sure this really shows any differentiation. I got the distinct impression the presenter was touting the new JavaScript based solutions – but it felt like a personal preference as opposed to a true recommendation.

Ben Taylor

Out of the box functionality for exporting maps coming 10.1.  I know there is the ability to export directly to pdf, I’m not sure yet what other file types will be supported. Custom symbolization of individual service layers will also be supported. No need to have to modify the mxd/msd and republish or create a custom solution if you want to change a service layer’s symbology.

I took in a session on .NET/Java Add-ins for Desktop. The coding of add-ins is a bit simpler, no managed com is required.  The add-ins are typically easier share and deploy. You can save your add-ins to a common directory and have your users install them from there. You also have the ability to digitally sign the add-ins to take care of any security woes.

A few positive changes to the Javascript API 2.2 release. The base class for the info window is now exposed. So you can extend and customize your own info windows much easier. There are a few much needed map events added to the API. In particular, an “on update start” and “on update end” event is now wired into the map. So you no longer have to listen to each layer’s event to determine what state your map is in. On the editing front, a new client-side method is exposed to check for self-intersecting polygons. You no longer have to waste calls to the server to simplify each edit to a polygon. Overall, there seems to be a lot of buzz about this new release.

Patrick Barnhart

The first day of my first trip to the ESRI Developer Summit was a good one.  This was actually the second day of the summit, the first day was taken up by layovers, delayed flights and general unhappiness but I arrived safe and, due to the weirdness of flying across a bunch of time zones, in time to get plenty of rest for the next day.

We began with the Plenary Session (which I had to look up, it means all members of all parties in attendance, if you’re curious) which was really the highlight of my day.  This session was a broad overview of all the new developer toys either recently added to ArcGIS 10 or that are on the way with 10.1.  There was an abundance of great stuff, to the point where I was getting somewhat overwhelmed by it all about three quarters of the way through, but looking back at the end of the day a couple of things stick out.

‘The internet as an operating system’ – so with the cloud and distributed computing I’m aware of the move towards having lightweight computers, basically keyboards and monitors, with the muscle being provided by a server living out in a farm somewhere but this is the first time I’ve heard it phrased in this way.  It also goes along with ESRI’s push towards hosted mapping solutions on ArcGIS.com, one of the speakers said right out that one of their goals is to have organizations use ArcGIS.com instead of having their own copy of ArcGIS Server, which is surprising to hear from someone in the business of selling software.   You can publish, edit and analyze map data online and they’re adding tons of widgets to make the whole process easy enough that pretty much anybody can do it.  I’m not going to try and go through all the functionality they’re adding but it feels like to me like they’re really committed to this approach and I’m a big fan.

Anybody remember having to actually write code?  The ESRI family of APIs (Mobile, Flex, Silverlight, Javascript, REST) seem to be doing all the work, a couple lines of code can do amazing things and a lot of the framework can be built in good ol’ wysiwyg editors using a mouse.  Once the bones are in place you still have to do some coding but this is a trend that’s been going on for a while and is still picking up speed.

10.1 is going to be awesome.  Yeah I know, they say that every time but really, an embedded web server, new and improved online printing capabilities, web accessible feature editing and symbology customization, the list goes on and on.  I won’t really be sold until I can do some of this stuff myself but the peek I got today fills me with warm fuzzies.  ArcGIS has always had power and tons of functionality but the words ‘heavy’ and ‘slow’ come up a lot and in addition to the new toys I saw some really, amazingly fast demos today.

Most of the other sessions I attended were about the Javascript and REST APIs but I’ll be going to the advanced version of these in the days to come and will talk about them then.  The coolest demo of the day award is a toss-up between drawing travel time/distance polygons on the fly (unbelievably fast) and using an XBOX Kinect for map navigation (it’s kind of rough at this point but a very interesting concept – once ESRI puts the sessions online be sure to check these out.

Tommy Bramble

Day 2 was kicked off with the Plenary Session.  This is basically the opening ceremony for the conference where the top Esri techs demonstrated their latest and greatest products and described the technological road ahead for the ArcGIS System.  A lot of information and cool technology was demonstrated in this all-morning session, but some of the key points I took away included;

  • ArcGIS products should now be considered wholly as a system, the ArcGIS System, where there is a focus on the cloud platform and services and data are not specific to any one component or device in the system.
  • ArcGIS Online (ArcGIS.com) is a powerful web tool for creating maps and sharing data and information.  ArcGIS online has been heavily integrated into most all components of the ArcGIS System, from ArcMap to the new Application Builder tools for flex and silverlight.  Esri is even implementing a new ‘portal’ feature that organizations can leverage to create a private ArcGIS Online space hosted by Esri or within the organization’s intranet.
  • ArcGIS Server 10.1 is fast!  The ArcGIS Server product manager demonstrated the new 10.1 server system by calculating drive times by mouse click, which took less than a second to render in the live demo.  Other wow factors included the new out-of-the-box high quality print tasks and feature service editing enhancements.
  • The new ArcGIS desktop runtime makes deploying custom desktop apps a lot easier.

Esri techs also really wowed the crowd by demonstrating some of their R&D work, a WPF-based map viewer that used Microsoft’s Xbox Kinect (and human movement) to control the map.  Not much utility for that now, but definitely was really cool, especially for an Xbox junkie like myself.

My first session of the day was ‘The road ahead for ArcGIS Server Developers’.  This was basically the detailed rundown on what changes will affect the AGS developers as they migrate to AGS 10.1.  Esri re-emphasized that developers discontinue their use of the WebADF, which is being deprecated, and move on to the WebAPI’s.  Esri is removing DCOM/AGS Local Connections in AGS 10.1, so if you are using local connections with the WebADF or making a local connection with ArcObjects your solutions will no longer work.  However, Esri states that if you are using the WebAPI’s you will be able to upgrade to 10.1 without anything breaking in your applications.  Optionally, you may want to update your applications to use the latest versions of the APIs, which will be updated to take advantage of any new features available in 10.1.  Some of the new features that will be added to AGS 10.1 services include;

  • Ability to add and remove layers to an existing map service using the new ‘workspace’ feature.  Basically, when you create a feature service you can specify a file-based directory where any data that resides in that directory can be dynamically added or removed from your feature service.
  • Ability to dynamically change layer symbology and transparency within your feature service.
  • Ability to change the geodatabase version being used (i.e. switch between versions to instantly see the different edits on your web map).
  • Ability to dynamically create new layers based on JSON markup (i.e. create a buffer result layer based on an existing feature layer and add that to your map).
  • The new high-quality print method that will be exposed through the updated WebAPIs.  The print task can also be extended using ArcPy and exposed as a geoprocessing service if you require more print options than will be exposed through the WebAPIs (i.e. print  resolution).

Also, AGS 10.1 will be native 64-bit runtime.  It will not run on 32-bit servers.

My second session of the day was ‘Using the ArcGIS Server REST API’.  This session was very good and presented well, but there wasn’t a wealth of new information here for the seasoned users and there was no discussion regarding any changes that could come with 10.1.  The session focused on current features available at AGS 10.0 and best practices.  Some new information I took away from this session included;

  • Using ETags and Cache-control headers with the REST requests/responses.
  • A review of the different service response formats.
  • The new operations for editing available at 10.0 that use HTTP POST.
  • A brief description of how to browse and consume the REST SOE’s, which are new to 10.0.

My third and final session of the day was ‘Extending ArcGIS Server Services using .NET’.  This session focused on using Server Object Extensions to extend/customize your AGS 10.0 map services.  Esri is highly recommending developers move to this design pattern now for their AGS custom solutions that are using ArcObjects in preparation for the major change that comes with AGS 10.1.  Again, with AGS 10.1 you will not be able to make local connections to AGS or use the fine-grained ArcObjects classes that you would create from your local connection.  Alternatively, you will have to build your custom ArcObjects solutions within the SOE framework at 10.1.  Some key information taken from this session includes;

  • SOE design pattern is available now at 10.0 and Esri recommends moving to this framework now if possible.
  • Research alternative solutions that may be available with 10.0 and 10.1 before developing your SOE.  There may be a new option available for solving your development issues (i.e. geoprocessing services or the new print tasks available at 10.1).
  • SOE’s described as fast, pluggable, and functional.
  • Ability to create SOAP or REST endpoints from your SOE, although Esri recommends REST due to its lightweight nature (and they only demo’d the REST solutions).
  • You must register your SOE to your AGS and then enable it for your map service through the ArcGIS Server Manager.  The SOE registration process is currently complicated so be sure to use the Resource Center and Help docs (including the sample solution for registering SOEs provided by Esri).
  • Also, make sure to take advantage of the Visual Studio SOE templates Esri has made available through the Resource Center.

In conclusion, it was a very good first full day of the summit with a ton of information to absorb.  Esri has done well so far and I’m looking forward to the REST of the conference.

Steve Mulberry

As always the Plenary Session showcased some exciting new technology and advancements with ArcGIS.

Some of the notable changes and improvements available now and coming in v10.1:

  • ArcGIS Server – Will become one component at v10.1, no more SOM/SOC which means no more managing of the 3 accounts (SOM, SOC, and Web Services). All communication is through HTTP/REST! J
  • ArcGIS Online – ArcGIS Explorer Online has a new face lift, which includes full support for tool tips or Popup windows, @ v10.1 support for hosted data/services as feature service or tiled (this means also online editing of the data you post), and its availability as a Portal you can run internally.
  • All the Web API’s are improved, look for the 2.2 beta releases on the resource.
  • New Application builder for Silverlight viewer.
  • New Flex Viewer configuration Builder…no more opening up the XML configuration file!
  • New ArcGIS Runtime, small foot print with simple copy paste deployment method. 64-bit support, new development architecture.

Overall the v10.1 is Server focused with faster native 64-bit support, dynamic legends, new print service, all HTTP/REST communication, dynamic rendering, and simple entry level into mapping through the spatial data services. New FGDB API, desktop SDE administration tools (No more command line J). Right click sharing/hosting from your ArcMap project.