Well, I finally figured it out with some more experimentation. I think maybe the answer is in the FDK docs, but it was elusive to me and my initial experiments failed for some reason. Pasted below is some info I sent to Maxwell Hoffman at Adobe, in case it would be of interest to their technical publications team:
RE: Adding a custom dialog box to a workspace
It took me a long time to figure this out because I think that the FDK documentation is missing some details. Here is what it says now:
"Because FrameMaker provides support for workspaces, the client's modeless dialogs can become a part of a workspace. To make this work, the client has to handle the notification FA_Note_Dialog_Create, which is sent to the client when the workspace has to launch the modeless dialog for a particular client."
I guess the answer is in there, but there was not enough clarification for me to figure it out initially. I finally figured out that when you save a workspace with the dialog open, the notification then happens automatically once you select or reset that workspace in the future. Also, there is the fact that "sparm" is set to the name of the dialog, which you need to test in order to know how to respond. So, I would recommend adding some verbiage similar to the following:
When a workspace is saved with a custom modeless dialog open, the name and positioning of that dialog are stored with the workspace. Then, when the user selects or resets the workspace, FrameMaker sends the FA_Note_Dialog_Create to the respective client with sparm set to the name of the dialog box. It is then the responsibility of the client to handle this notification and execute the required code to open the dialog box again. Once the dialog box is open and the client code exits, the workspace configuration routine will continue and automatically position the dialog box as stored in the workspace.
Note the following:
- FA_Note_Dialog_Create is only sent when the workspace includes a custom dialog that was open when the workspace was saved. If multiple custom dialogs were open, individual notifications are sent for each.
- The client notification code must reopen the dialog itself (F_ApiOpenResource(), etc.), then the workspace will position it.
- Like all notifications, FA_Note_Dialog_Create must be enabled with F_ApiNotification().