Your filter function will be called by the dialog manager. You don't really have to declare it to be anything in particular, you just have to make the function you want and pass its address into ModalDialog.
The Pascal declaration you have written tells us about how the stack will be set up when the filter proc executes. The stack frame (pointed to by A6) will consist of the previous A6 (at (A6)), then the return address (at 4(A6)) then itemHit (8(A6)) then theEvent (12(A6)), followed by theDialog (16(A6)) and room for a boolean result returned (20(A6)).
When your filter function executes, it can find its parameters using indirection from A6 as above. You can set it up like this:
; function MyFilter (TheDialog

ialogPtr;var MyEvent:EventRecord;var
; ItemHit;integer):boolean;
; return value is only 2 bytes, the rest are each 4 bytes long
myFilter:
; now some equates to find my parameters
itemHit SET 8
myEvent SET 12
theDialog SET 16
result SET 20
allparams SET 12 ; this is the total number of bytes in the parameters
; here you will make some equates for your local variables
myLocalLong1 SET -4
myLocalLong2 SET -8
myLocalRect SET -16
myLocalShort SET -10
; now you can set up your stack frame to include your local variables
LINK A6,#myLocalShort
; here you put your code to do whatever you want your filter function to do
; when you need them, your parameters and locals can be accessed relative to
; A6. For example, If I wat to know what dialog item was hit, I could call GetDItem
MOVE.L theDialog(A6),-(SP)
MOVE.W #buttonitem,-(SP)
PEA myLocalLong1(A6)
PEA myLocalLong2(A6)
PEA myLocalRect(A6)
_GetDItem
; at this point, the dialog item type is in myLocalLong1, a handle to the
; hit item is in myLocalLong2, and the items box rectangle is in myLocalRect
; now you do whatever you need to do.
; if your filter function actually handles the event, return some non-zero number
MOVE.W #$0100,result(A6)
; or if you didn't handle it you return zero
MOVE.W #0,result(A6)
; unlink and return
UNLK A6
MOVE.L (SP)+,A0 ; this gets the return address from the stack into A0
ADDA.W #allparams,SP ;this unloads the parameters put there by the caller
JMP (A0) ; all done