FileMaker Fever

FM7 Migration Bible Available

The FileMaker consulting and development community has been waiting in anticipation for this promised tech brief from Danny Mack et. al. called FileMaker Pro 7: Migration Foundations and Methodologies (12.4 mb). I've been checking periodically at FileMaker Inc's FileMaker 7 Upgrade Tech Briefs page for over a month - it was *due* in early April - and finally found it this morning.

It's big. It's 140 pages long plus an essential 56 page appendix called Conversion Issues and Resolutions. Right now, this is the bible for upgrading to FileMaker 7. If you have a FileMaker system that is important to you and you are thinking about upgrading, make sure that you or your consultant/in-house expert sets aside the time - and reads this.

The book is divided into three sections: Foundations, Methodologies and the Appendix. Foundations is a series of articles or chapters written by Danny Mack and other leading lights in the FileMaker community. Methodologies is less conceptual and more practical but also strategic. Here's the Overview from the book:

FOUNDATIONS

Leveraging the Value Proposition of FileMaker Pro 7 – Michael Thompson
Michael discusses the advantages of FileMaker Pro 7 from the point of view of both developers and business owners, and addresses the strategic issues associated with migrating to this new technology, including how to recognize a return on the investment in migration.

The FileMaker 7 Relational Model – FileMaker, Inc. and Danny Mack
A fundamental article of the new relational model, including a comparison to earlier versions of FileMaker Pro, an introduction to the relationships graph, and an explanation of “context”. It illustrates the new features and the new rules with easy to follow examples.

File References in FileMaker Pro 7 – Corn Walker
File references are viewable and editable in FileMaker Pro 7 and the issues associated with them are fundamental to getting your converted solution up and running successfully.

Scripting Issues Encountered When Migrating to FileMaker Pro 7 – Darren Terry
In converted solutions, certain scripts may not function as they did previously. Darren reviews these issues in detail so that you understand how the behavior has changed, and what to do about it.

Security and Access Privilege Issues – Steven Blackwell
Passwords and groups are converted to accounts and privilege sets, but the rules have changed. Understanding the details of access privilege conversion is essential to replicating the original behavior or extending it.

“Record Ownership” in Converted Solutions: Opening and Committing Records – Ilyse Kazar
The script steps and other events that cause records to be opened (locked) and committed (unlocked) have changed signifi cantly. This has major implications for the behavior of converted solutions as well as for new solution design.

Migration and Web Publishing – Bob Bowers
This is a primer on the realm of FileMaker 7 web publishing, including both Custom Web Publishing and the entirely new Instant Web Publishing.

METHODOLOGIES

Conversion Basics – Danny Mack
Danny presents a step-by-step orientation to the conversion process, including preparation, testing, and the necessary fundamental tasks for getting your solution “restored” to its original functionality.

Adding a New Interface File to an Existing Solution, Later Consolidating Tables – Todd Geist
How does one migrate a solution, most reliably and economically, to an optimal FileMaker Pro 7 architecture? Todd lays out a step-by-step migration strategy, including a rationale that may leave you thinking: “why would you do it any other way?”

The Separation Model: A FileMaker Pro 7 Development Model – Colleen Hammersley & Wendy King
FileMaker Pro 7 is a dream come true for those who have long advocated the separation of data and interface. Colleen and Wendy present the new application model in all its glory.

Bridging .fp5 and .fp7 – Ernest Koe
There are many real world scenarios in that it will be necessary for FileMaker Pro 7 files to exchange data with FileMaker .fp5 files. There are several technologies that make this possible.

APPENDIX

Conversion Issues & Resolutions – Team
This is the documentation of specific issues that can be encountered in solutions that have been converted to FileMaker Pro 7, and suggested resolutions (what you can do to replicate the original behavior). The issues are cross-referenced to the behavior changes in the application documented in the .pdf entitled “FM 7 Converting Databases”.

Could FileMaker Talk to Google and Mapquest?

Today I was preparing a mailing to all of my FileMaker clients. I wanted to warn them about the fact that you can't just upgrade to FileMaker 7 from previous versions without giving it a second thought. One of my customers called today and said she bought FileMaker 7 because she upgraded to OS X. I told her that she should return it because the FileMaker 6 she already had would work fine in OS X and unless she wanted to spend some money on tweeking before and after, the system I built for her was going to break in 7.

This was not the first customer doing a FileMaker 7 upgrade *on-the-fly* without consulting me first. So, to avoid further problems, I wanted to do a mailing to all my past and present FileMaker customers. Since I haven't worked with some of my customers for few years, I don't have all their email addresses. In many cases, their phone numbers had changed too so I couldn't call to get their email addresses.

Google.
This is when I thought that it would be great to have a *Google* button right in my FileMaker database that could Google anyone's full name or company name with one click. If I could do that, then the next step would be to create a script and button that could Google a set of records and return data to me on what it found. I'm guessing that I will be able to create a Google button for one person. And that's probably as far as I'll go for now, but if someone wanted to build a little plug-in or custom function that would Google a FileMaker Found Set and return records, I'm ready to plunk down the cash!

Mapquest.
Later, I was looking for a great restaurant to take my Dad to for his birthday. I got a good recommendation after trying Google and not really finding what I wanted with assurance it would be good. So I wanted a map for my sister and went to Mapquest. I had already entered the restaurant into my database for future reference and actually dragged the Mapquest map right into a Map field I had available in the database.

That worked, but why couldn't I have a Mapquest button that would go to Mapquest, get the map and put it in my database for me based on the address that was in the database already? That would be so cool. In this case, if I can do it for one contact, I can definitely run a script that would put maps into every record (thousands) in my database. Learning about web services and FileMaker is starting to seem like a great idea.

Anyone want to build something like this? It's relatively simple if Mapquest has an open API and I'm sure tons of FileMaker users would be interested.

FileMaker 7 Security is Sweet

After screwing up my courage and reviewing my error list from MetaDataMagic today, I decided to just go ahead and convert my files and see what happens. Looking at MDM's conversion issues report and looking at my Studio Manager database, I could see that I'll need to check a lot of things systematically but the nature of the problems is such that it might work to make the adjustments after conversion.

This breaks the general guideline suggested by FileMaker 7 migration guru, Danny Mack, but I'm planning on multiple trial conversions of my product anyway and wanted to see what would happen when I just threw the FileMaker 7 conversion switch.

First, the good news. The conversion took less than ten minutes. Nothing blew up. I didn't need a fire extinguisher or anything. Remember, I did clean up the file references before converting and ran MetaDataMagic on my files and reviewed the error list and recommended fixes. For each recommendation, it tells you whether you should make the changes before or after you convert or that you can do it either way.

I have 4 levels of groups and passwords in my Studio Manager files. I wondered what would happen with them. My first experience was good. I was able to use my master password to get into my files after the conversion, using the master password for both the account name and the password as I had read I should. Whoopie!

My luck didn't last long, though. When I clicked on a script-protected tab or file navigation button, I got my self-authored "sorry, access denied" message even though I was logged in with the master password. I wasn't surprised - I wasn't expecting miracles. I just went in and looked at my security calculations and security checking scripts looking for a culprit and couldn't figure it out.

Next, I changed the master-access account name (formerly the group name) to "master" to match my security calculations but that didn't work. The calculations were actually using the privilegesetname not the account name, but I was still getting my sea legs and wasn't crystal clear on the distinction yet.

Notice that I was focusing my attention on technologies I was familiar with. Finally, I got smart and cracked open the new Define Accounts & Privileges command. There I saw what some of you already know - the master access privilegesetname is hard-coded to be "Full Access". So I just went around and changed my security calculation fields to use Full Access instead of master and things started working!

One other little problem occurred around this. When I opened my files for the first time in FM7, I was offered the option to remember the passwords and I said yes to all. Seemed like a good idea at the time. But then I ran into the problem that I had not entered the master password in one of the files which has a special secret password and now it was set to automatically open with a lower level password by my OS X keychain.

Not to worry, I looked up keychain in the help and found the keychain entry for the file I was having a problem with and after futzing around for a while just deleted it. That worked! This time when I tried to open the file, I got an account and password prompt and could enter the high-end account and password. Good to go!

Now that I've conquered these little hiccups in my security learning curve, I am really getting psyched. I know how incredibly easy the keychain stuff is. Opening my FM7 Studio Manager files will be effortless. My system password protects all these saved account names and passwords along with all the other passwords I have. Thanks, FileMaker Inc., for making your password and privileges system compatible with the OS X keychain! FileMaker Inc. is owned by Apple, so I guess it's about time, but thanks anyway!

For years I've steeled myself to the fact that the password and security part of FileMaker has been convoluted and unintuitive. It was the one thing in FileMaker I hated to use. Now, even though the new FileMaker 7 security is much more powerful, it's easy. It's straight forward. For a FileMaker geek like me - it's fun!

MetaDataMagic 2 Roadtest, Part 2

MetaDataMagic 2 rocks bigtime! I did two new things with it since my last report:

1. Fixed the File References in my product.
2. Built a Conversion Issues report for it.

Both of these functions are excellent with the second providing more value to me than the first.

FIXING FILE REFERENCES
Prior to FileMaker Pro 5.5, you couldn't specify that you only wanted relative file references which are great because if all your files reside in one folder, then you can move that folder anywhere and none of the file references break. Prior to 5.5, less robust file references like IP addresses were stored and, worse, the references weren't directly accessible for modification. They are in FM7, but even so, the manual method can't compete with MDM's file reference powertools.

There's an *auto-fixer* that will eliminate all references to files outside your solution *and* convert and set your file references to *relative path only* - all the rest go away. I ran auto-fixer without a hitch in less than 2 minutes on my complex 27-file Studio Manager product. That's time savings!

I had tons of file references in these files and was left with the minimum I need to have things work correctly and as fast as possible. I'll let you know when I try the performance out how much performance gain I get in FM6 and FM7.

FINDING CONVERSION ISSUES
Whoa! This thing is amazing! New Millenium has identified a total of 82 different possible issues so far. I had 23 of them applicable to my solution. Of those 23, 18 were considered high impact and 7 medium impact. You get a brief description of the element that is causing the issue and then a potentially multi-paragraph description and pointers directly to the files impacted in your solution so you can go fix them right now.

I was tempted and on the verge of converting my Studio Manager files to FM7 without doing much cleanup in the hopes everything would be OK until I ran this report. Here's just the first 3 of 23 conversion issues found:
(42) Go to Related Record button function
(216) Go to Related Record Script step
(25) External subscript transfer focus to another file

Ack! Those numbers in parentheses beside each issue are the number of occurrences of that issue in my files. I've got some serious work to do. It will be easy work with this information and tool but it will be a lot of work.

Luckily, I will at least have a hope in hell of doing the fixes in an organized way and I can run the report on my files to see where I stand in handling the issues. There's a checkbox that lets me tell it to ignore that issue for now which I bet will come in handy.

FREE STUFF ALERT
You get the Conversion Issues database free when you download the trial version of MDM2. There's a lot of information in there. Go forth and download!

MetaDataMagic 2 Roadtest, Part 1

I've run three complete FileMaker 6 databases through MetaDataMagic 2 with no problem in the last 48 hours. The biggest one so far, my Studio Manager product, took about an hour to process with its 27 files and tons of fields, layouts and scripts. I was quite pleased with the level of detail of information I got and the ease of getting it.

I love that I don't just see a progress bar during the analysis but instead a list of records with time elapsed as each analysis task is completed on each file. There are 27 analysis tasks. Some tasks are quick and others take longer. The longest task in most situations is going to be *get layout item data* - that's the step where every layout object is itemized and described down to it's exact position on the layout.

One of the most interesting pieces of information I found out about my Studio Manager product was how many times files had not been closed properly. Many of these files have been around for several years and I could scan the list to see how much wear and tear they have endured. Good to know. In one file the number of improper closes was 20. Since I'm having no problems with it, I won't do anything about it right now, but this information would go into my decision-making for whether or not I would want to rebuild it from scratch in FileMaker 7. I like this information!

The process. Here's what I did to get started.

1. Went to New Millenium's website and downloaded MDM using their shopping cart


2. Cracked open the ReadMe file for basic operating instructions.


3. Opened the main FileMaker file.


4. Decided which system to analyze first.


5. Clicked the *Process a New Solution...* button


6. Selected the folder to process, entered master passwords and clicked *Go*


7. Immediately up came a second window with records listing progress and duration


8. I watched most of it go because it was fun and then did something else while waiting for completion


9. Then I jumped to looking at any error listings.


10. Next stop was the Files Not Closed Properly list...


11. Later, I wanted to do a 2nd analysis for files for one of my clients.


12. To do that, I just duplicated the folder for the first analysis, renamed it and started MDM up again.


13. Then I clicked the *Delete all Metadata* button.


14. Then clicked *Process a New Solution...*

I needed one of my analyses for a specific purpose: to give my client a correct list of relationships in all the files so that she could double-check them to make sure they were correct. She had opened files from a remote machine when only some of the files were open on the server and some of the relationships got misdirected. I was able to go to the Relationships file for her analysis and quickly send her a pdf by email.

More soon as I use MDM 2 for helping me analyze my product's readiness for conversion to FileMaker 7!

Just bought MetaDataMagic 2

I just completed my purchase and download of version 2.0 of MetaDataMagic by New Millenium. Version 2, released last week on March 30th, is an essential tool in migrating your existing FileMaker files to FileMaker 7. Here's why:

1. It incorporates Danny Mack's new Conversion Issues Reporting Tool and as it creates its detailed analysis of your files, it also records any conversion issues it finds (Danny is the author of a 27 page document on FM7 migration that is featured on the FileMaker, Inc site right now and the author of FileMaker 7 Migration Foundations and Methodologies that is due out any day now - he gave an excellent presentation in Santa Clara on behalf of FileMaker, Inc. regarding FM7 migration in February).


2. It includes a File References Fixer that streamlines the process of eliminating extraneous file references in your files so they don't slow things way down when you convert to FM7. The File References Fixer will also help you clean up FileMaker Pro 6 files to their benefit.


3. It includes their Conversion Log Analysis Tool - which helps you analyze the conversion log for errors

Maybe you are still on the fence - trying to decide whether you want to put your money into this product. I was convinced by the essential migration features, but I'm also really looking forward to the extra system analysis documentation tools not available elsewhere such as:

• Relationships: sort spec, with fields and value lists.
• Layout items: text blocks, graphics, and buttons, including coordinates
• File stats: times not closed properly, times recovered

I expect to run MetaDataMagic on at least one of my FileMaker solutions in the next 24 hours. So, stay tuned for my first person reports. ;-)

FMPro Migrator vs. FMrobot

Just days after FMrobot came out (see my last post), .com Solutions Inc. updated FMPro Migrator to include the ability to convert multiple FileMaker 3, 4, 5 or 6 files to multiple *tables* within a single FileMaker 7 file. And it only costs $99.

Keep in mind as we go through this comparison that FMPro Migrator does many other kinds of migrations (to MySQL, Oracle, Access, SQL Server, DB2 and Sybase), so its ability to consolidate multiple FileMaker files into a single FM7 file is just one of its many features. FMrobot, on the other hand, is by comparison a one-trick pony. It focuses only on migration to FileMaker 7 and associated functions.

FMrobot, at $199, is twice the price of FMPro Migrator. Bottomline, FMrobot offers at least twice the value of Migrator's FM7-related capabilities. The serious developer and anyone else who has a big need for migration (read lots of files and/or files with lots of fields), will pay the extra money for FMrobot. Besides the extra $100 for FMrobot, you'll also need FileMaker Developer 6 or 7. Most serious developers and corporate FileMaker installations already have a copy of Developer which is $499 vs. FileMaker Pro's $299. FSA Associates and Partners get Developer as part of the FSA membership - $595 a year.

I started with FMPro Migrator because .com had an OS X version and FMrobot only runs on Windows at this point - an OS X version is in the works. I hadn't read the Migrator sales materials on the website very closely and didn't realize that I still needed to run the second half of the migration process on a PC or Virtual PC because Migrator uses ODBC and the ODBC plug-ins for FM 7 don't work on the Mac yet. Disappointment #1.

After running FMPro Migrator on my 48 open FileMaker 6 files which created all sorts of CGI scripts, I copied the CGIs to my PC and then downloaded Perl on my XP machine and installed it - I was directed to a free download. Then I had to install a FileMaker ODBC driver which was on my FileMaker 7 CD. I got stuck activating the ODBC driver and emailed .com Solutions and received a reply from David Simpson, the man behind .com Solutions, at 10 pm Friday night. Not bad - he answered my questions and offered to help me by phone on Saturday! I sent a few more emails that night. David called me on Saturday morning. He spent over an hour with me walking me through installation issues, some problems I had converting one of the simpler files and giving me some PC tips. He's very helpful and patient.

When I ran the CGIs for my 48 files, 23 of them were created with no problem. I was disappointed again, though, even with the files that converted easily and very quickly (less than a minute per file). All I got was text, number, date, time and container fields. Calculations just came out as static text, number or date fields. Ouch! Further, no entry options made it through the translation process and all field names got forced to lower case and spaces were replaced with underscores.

That left me with 25 more files to investigate and *fix* one by one. I spent another hour and got about three more files through the process. At this point, I decided it was time to try FMrobot! Remember that I bought FMPro Migrator the day after it was released. Field name filtering had not yet been perfected. Documentation wasn't sufficient - at least for me. I wouldn't be surprised if the conversion success ratio is already much higher by now. By Monday a QuickStart manual had been posted to the .com Solutions site that would probably have allowed me to handle the PC part of the process with dispatch except perhaps for CGI failures that may still have occurred in some cases.

Then I tried FMrobot. As soon as I tried it, I stopped all further use of Migrator. No contest. Besides its ability to smoothly build tables in a FileMaker Pro 7 database from a FileMaker Pro 6 Database Design Report with entry options and calculations (even though lots of the calculations were commented out), it does more. It provides a way to duplicate a table within an FM7 database. It provides us with the ability to move tables from one solution to another. FileMaker consultants and developers can build utility tables of various sorts and sell them because they can be copied into a customer solution (since layouts and scripts don't come along for the ride, there would be some limitations). A FileMaker front-end for FMrobot is in the works and will be a free upgrade.

Hopefully, these upstart migration utilities that have been released within 30 days of the release of FileMaker Pro 7 will be improved and extended in many ways in the next weeks and months. I suspect that we may see additional entrants in this market as well because the need is great and universal for anyone with a sizeable multi-file FileMaker Pro solution. I wouldn't even be shocked if FileMaker, Inc. further improves its conversion routines to help us out in a few months. It will get easier over time as utilities improve and we learn more about the best migration techniques.

Migrating to FM7 with FMrobot

You don't need FMrobot to move your FileMaker 6 and below files to FileMaker 7, but you sure will be happy if you have it. Although groundbreaking in many ways, FileMaker Pro 7, has one glaring weakness - it doesn't convert files created in previous versions of FileMaker Pro cleanly and completely. And one thing it just doesn't even try to do is take a database with multiple files and combine it together under one roof as multiple tables inside of one file - in most cases the most desirable set-up.

If you want some or all of your files under one roof, you are faced with rebuilding all but one of the files from scratch. You do this by converting all your FM6 or earlier files to FM7. If you start with 10 files, you'll get 10 files when done converting to FM7. Let's say for argument's sake that a Contacts and Invoices file are among your 10 files. Name the Contacts file something like Company Database. Now create the Invoices file and all its fields one at a time as a new table inside the Company Database. Make sure to create all the calculations (you can copy and paste from your old file if you like going back and forth between files for each individual field). This is where FMrobot saves the day.

New Millenium has built FMrobot ($199), an automation tool that will rebuild a file that resides outside the walls of your single-file database and put it inside. It works only in Windows for now. It works by automating keystrokes and will rebuild a file before your eyes field by field including calculations, entry options and lookups. If it runs into something that produces an error in a field and it will, it just comments the calculation out but preserves it so that you can make what is usually a small change to get the calculation to work.

I bought FMrobot about ten days ago to help me convert my 28-file business database running in FileMaker 6 into a single file, 28-table FileMaker 7 database. What a life-saver. FMrobot draws upon the Database Design Report that is generated by FileMaker Developer 6 or FileMaker Developer 7 which lists for $499 and is free to Associate and Partner members of the FileMaker Solution Alliance. FileMaker Developer is a full version of FileMaker, so you pay a $200 premium for its extra capabilities.

Once FMrobot has done its magic and you have all the files built. You'll need to fix some of the calcs and will also need to build or copy layouts into these files and import scripts. You'll then need to fuss a bit with these layouts and scripts to get them working exactly as they did before.

What I did to make it worth my while is I streamlined my system that had been built over many years and had lots of junk in it - extra fields and layouts no longer used. Now that I had been using the system for years, I knew which parts of the system worked well and which seemed like it would be a good idea but really never got much use.

This partial rebuild process starts to become worthwhile if you take advantage of it as an opportunity to take a fresh view of your system and build it better this time. Of course, because FileMaker 7 is so new, your first cut at a FileMaker 7 version won't always fully and best exploit its new features and differences from previous versions of FileMaker.

As a serious FileMaker developer, I'll be learning everything I can as fast as I can about FileMaker Pro 7. Stay tuned as I record my discoveries here.

FileMaker Timeline

I just looked up FileMaker in the Wikipedia for the first time and found, among other good things, a chronology of all FileMaker releases. It's a good tidbit. I did a bottomline edit to just show the major releases here:

Glenn Koenig has written a fascinating behind the scenes story about the early years of FileMaker based on a phone interview with one of the original four founders of Nashoba Systems, Spec Bowers, called The Origin of FileMaker Pro.

2 Free FileMaker 7 Videos

Over at the excellent FileMaker Magazine site run by one of my all time favorite FileMaker gurus, Matt Petrowsky, they've posted two great introductory videos about FileMaker 7.

An Introduction to FileMaker Pro 7

An Introduction to FileMaker Pro 7 - Part 2

If you are wanting to see what all the excitement is about in a live demo, you must see these two cool 40 and 50 minute videos. Matt provides running commentary while you watch him working with FileMaker Pro 7. Double-click if you need to pause and then double-click again to unpause. Matt does a great job - he's taught standing room only sessions at the FileMaker Developer conference for years and is the author of Scriptology.