Creating a new DDM fitting function

Here, we are going to go over how one would add a new fitting function. The function we will add is one which models a polydisperse collection of objects. We will be fitting the DDM matrix to the function:

\[D(q,\Delta t) = A(q) \left[ 1 - \exp \left( -\frac{\Delta t}{\tau(q)} \right) \left( 1 + \frac{\mu \tau^2}{2} \right) \right] + B(q)\]

So we will have four fitting parameters: \(A\), \(\tau\), \(\mu\), and \(B\). The relative polydispersity is given by \(\mu \tau^2\).

Importing the necessary modules

[1]:
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np #numerical python used for working with arrays, mathematical operations
import xarray as xr #package for labeling and adding metadata to multi-dimensional arrays
import sys
sys.path.append("../../PyDDM") #must point to the PyDDM folder
import ddm_analysis_and_fitting as ddm
import ISF_and_DDMmatrix_theoretical_models as models

Creating the function

We will first create the function for this new model in the file ISF_and_DDMmatrix_theoretical_models.py.

The first argument of this function will be the lagtime. The other arguments will be the four parameters in the model.

[2]:
####################################################################################
# THIS FUNCTION TO BE PLACED IN THE FILE `ISF_and_DDMmatrix_theoretical_models.py` #
####################################################################################

def dTheoryPolydisperse_DDM(lagtime,amplitude,tau,mu,bg):
    r"""Theoretical model for the  DDM matrix to account for polydisperisty

    Parameters
    ----------
    lagtime : array
        1D array of the lagtimes
    amplitude : float
        Amplitude, "A" in equation below
    tau : float
        The characteristic decay time
    mu : float
        To account for polydispersity
    bg : float
        Background term, "B" in equation below

    Returns
    -------
    ddm_matrix : array
        DDM matrix as shown in equation below

    Notes
    -----
    This model is used when polydispersity is present.

    .. math::
        D(q,\Delta t) = A(q) \left[ 1 - \exp \left( -\frac{\Delta t}{\tau(q)} \right) \left( 1 + \frac{\mu \tau^2}{2} \right) \right] + B(q)

    """
    relative_polydisp = mu * tau * tau
    isf = np.exp(-1 * (lagtime / tau)) * (1 + (relative_polydisp/2.0))
    ddm_matrix = amplitude * (1 - isf) + bg
    return ddm_matrix

Creating the parameters dictionary

Next, we will create a dictionary in the file fit_parameters_dictionaries.py.

[3]:
ddm_matrix_polydisperse = {}
ddm_matrix_polydisperse['parameter_info'] = [
        {'n': 0, 'value': 0, 'limits': [0,0], 'limited': [True,True],
         'fixed': False, 'parname': "Amplitude", 'error': 0, 'step':0},
        {'n': 1, 'value': 0, 'limits': [0,0], 'limited': [True,True],
         'fixed': False, 'parname': "Tau", 'error': 0, 'step':0},
        {'n': 2, 'value': 0, 'limits': [0,0], 'limited': [True,True],
         'fixed': False, 'parname': "Mu", 'error': 0, 'step':0},
        {'n': 3, 'value': 0, 'limits': [0,0], 'limited': [True,True],
         'fixed': False, 'parname': "Background", 'error': 0, 'step':0}]
ddm_matrix_polydisperse['model_function'] = models.dTheoryPolydisperse_DDM
ddm_matrix_polydisperse['data_to_use'] = 'DDM Matrix'

Going over this dictionary

  • We first make an empty dictionary (we call it whatever we want):

ddm_matrix_polydisperse = {}
  • We then create a list of dictionaries using the key parameter_info. The number of elements in this list should be equal to the number of fitting parameters. For this new model we are implementing, we have \(A\), \(\tau\), \(\mu\), and \(B\). Each element in this list looks something like this:

{'n': 0, 'value': 0, 'limits': [0,0], 'limited': [True,True],
         'fixed': False, 'parname': "Amplitude", 'error': 0, 'step':0}

Make sure to enter in the parameter name in for the key parname. Here, we have parameter names of Amplitude, Tau, Mu, and Background. Also, the order of the paramters in parameter_info should match the order in which the arguments to the function dTheoryPolydisperse_DDM appear.

  • We then provide the function which we made and placed in the file ISF_and_DDMmatrix_theoretical_models.py in the previous step:

ddm_matrix_polydisperse['model_function'] = models.dTheoryPolydisperse_DDM
  • Finally, we specify whether the data to be fit is the DDM matrix or the ISF:

ddm_matrix_polydisperse['data_to_use'] = 'DDM Matrix'

Adding dictionary to the fitting models

We next must add this dictionary that we created (ddm_matrix_polydisperse) to the dictionary fitting_models in the file fit_parameters_dictionaries.py.

Note that the key used for the dictionary fitting_models will be how we will refer to the model in the YAML file. Here, we are choosing to use the key ‘DDM Matrix - Polydisperse’.

[4]:
fitting_models = {}
fitting_models['DDM Matrix - Polydisperse'] = ddm_matrix_polydisperse

Trying out the new model!

[13]:
import yaml
ddm_analysis_parameters_str = """
DataDirectory: 'C:/Users/rmcgorty/Documents/GitHub/DDM-at-USD/ExampleData/'
FileName: 'images_nobin_40x_128x128_8bit.tif'
Metadata:
  pixel_size: 0.242 # size of pixel in um
  frame_rate: 41.7 #frames per second
Analysis_parameters:
  number_lag_times: 40
  last_lag_time: 600
  binning: no
  overlap_method: 1
Fitting_parameters:
  model: 'DDM Matrix - Polydisperse'
  Tau: [1.0, 0.001, 10]
  Mu: [0.4, 0.001, 100]
  StretchingExp: [1.0, 0.5, 1.1]
  Amplitude: [1e2, 1, 1e6]
  Background: [2.5e4, 0, 1e7]
  Good_q_range: [5, 20]
  Auto_update_good_q_range: True
"""
parameters_as_dictionary = yaml.safe_load(ddm_analysis_parameters_str)
[8]:
ddm_calc = ddm.DDM_Analysis(parameters_as_dictionary)
Provided metadata: {'pixel_size': 0.242, 'frame_rate': 41.7}
Image shape: 3000-by-128-by-128
Number of frames to use for analysis: 3000
Maximum lag time (in frames): 600
Number of lag times to compute DDM matrix: 40
Using the full frame, dimensions: 128-by-128.
[9]:
ddm_calc.calculate_DDM_matrix()
The file C:/Users/rmcgorty/Documents/GitHub/DDM-at-USD/ExampleData/images_nobin_40x_128x128_8bit_ddmmatrix.nc already exists. So perhaps the DDM matrix was calculated already?
Do you still want to calculate the DDM matrix? (y/n): y
2022-02-15 11:44:34,640 - DDM Calculations - Running dt = 1...
2022-02-15 11:44:41,652 - DDM Calculations - Running dt = 5...
2022-02-15 11:44:45,105 - DDM Calculations - Running dt = 9...
2022-02-15 11:44:48,323 - DDM Calculations - Running dt = 16...
2022-02-15 11:44:52,230 - DDM Calculations - Running dt = 27...
2022-02-15 11:44:55,881 - DDM Calculations - Running dt = 47...
2022-02-15 11:44:59,466 - DDM Calculations - Running dt = 81...
2022-02-15 11:45:04,809 - DDM Calculations - Running dt = 138...
2022-02-15 11:45:07,986 - DDM Calculations - Running dt = 236...
2022-02-15 11:45:11,031 - DDM Calculations - Running dt = 402...
DDM matrix took 39.641406536102295 seconds to compute.
 Background estimate ± std is 211.17 ± 1.49
[9]:
<xarray.Dataset>
Dimensions:           (lagtime: 40, q_y: 128, q_x: 128, q: 64, y: 128, x: 128, frames: 40)
Coordinates:
  * lagtime           (lagtime) float64 0.02398 0.04796 0.07194 ... 12.59 14.36
    framelag          (frames) int32 1 2 3 4 5 6 7 ... 308 352 402 459 525 599
  * q_y               (q_y) float64 -12.98 -12.78 -12.58 ... 12.37 12.58 12.78
  * q_x               (q_x) float64 -12.98 -12.78 -12.58 ... 12.37 12.58 12.78
  * q                 (q) float64 0.0 0.2028 0.4057 0.6085 ... 12.37 12.58 12.78
  * y                 (y) int32 0 1 2 3 4 5 6 7 ... 121 122 123 124 125 126 127
  * x                 (x) int32 0 1 2 3 4 5 6 7 ... 121 122 123 124 125 126 127
Dimensions without coordinates: frames
Data variables:
    ddm_matrix_full   (lagtime, q_y, q_x) float64 175.0 193.1 ... 172.1 186.3
    ddm_matrix        (lagtime, q) float64 0.0 295.8 314.5 ... 206.3 208.3 207.5
    first_image       (y, x) float64 128.0 149.0 173.0 ... 175.0 229.0 215.0
    avg_image_ft      (q) float64 0.0 1.293e+05 5.225e+03 ... 105.3 104.7 105.3
    num_pairs_per_dt  (lagtime) int32 300 300 300 300 300 ... 289 283 275 267
    B                 float64 211.2
    B_std             float64 1.491
    Amplitude         (q) float64 -211.2 2.585e+05 1.024e+04 ... -1.699 -0.52
    ISF               (lagtime, q) float64 0.0 0.9997 0.9899 ... -0.6879 -6.08
Attributes: (12/16)
    units:             Intensity
    lagtime:           sec
    q:                 μm$^{-1}$
    x:                 pixels
    y:                 pixels
    info:              ddm_matrix is the averages of FFT difference images, r...
    ...                ...
    pixel_size:        0.242
    frame_rate:        41.7
    number_lag_times:  40
    last_lag_time:     600
    binning:           no
    overlap_method:    yes
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_14_4.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_14_5.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_14_6.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_14_7.png

Initiazing DDM_Fit class and fitting our data to a model

[16]:
ddm_fit = ddm.DDM_Fit(parameters_as_dictionary)
Initial guess Minimum Maximum
Amplitude 100.0 1.000 1000000.0
Tau 1.0 0.001 10.0
Mu 0.4 0.001 100.0
Background 25000.0 0.000 10000000.0
Loading file C:/Users/rmcgorty/Documents/GitHub/DDM-at-USD/ExampleData/images_nobin_40x_128x128_8bit_ddmmatrix.nc ...
[17]:
fit01 = ddm_fit.fit(name_fit = 'fit01')
Fit is saved in fittings dictionary with key 'fit01'.
q Amplitude Tau Mu Background
0 0.000000 1.000000 10.000000 0.003189 0.000000
1 0.202840 15.918614 4.263357 18.534856 3400.434926
2 0.405681 30.332247 3.518539 25.931651 5629.668970
3 0.608521 57.581148 2.961575 33.770350 9265.281048
4 0.811362 190.065873 2.145075 32.532448 14967.489051
5 1.014202 473.955467 1.351004 40.002862 17802.347026
6 1.217043 957.582514 0.975331 39.887403 18574.906136
7 1.419883 1549.598673 0.744532 42.757955 18671.705002
8 1.622723 2291.933349 0.550475 47.784090 16934.983376
9 1.825564 2975.005723 0.467657 56.028242 18384.154790
10 2.028404 3202.095113 0.404070 75.240054 20051.371018
11 2.231245 5266.187548 0.323464 81.886048 22482.548046
12 2.434085 7465.115533 0.294848 81.465499 26315.586325
13 2.636926 14184.070460 0.249017 59.968763 25765.143401
14 2.839766 16690.023915 0.220707 63.960972 25534.842535
15 3.042607 17804.609612 0.196756 74.491651 25061.623872
16 3.245447 17470.393043 0.172426 93.954449 23612.224217
17 3.448287 18728.923997 0.153925 80.094564 17048.030652
18 3.651128 15815.313807 0.137660 89.438046 12612.634732
19 3.853968 12046.238179 0.128386 94.163541 8782.300172
20 4.056809 8877.497642 0.120005 90.383462 5523.729980
21 4.259649 6030.477308 0.111387 89.458074 3255.010942
22 4.462490 3862.696157 0.103704 87.048636 1833.963963
23 4.665330 2332.250996 0.101411 91.527842 1167.036548
24 4.868170 1375.128511 0.101480 76.471608 700.101962
25 5.071011 838.960280 0.101001 69.530588 472.589432
26 5.273851 582.387628 0.099502 66.587505 379.680374
27 5.476692 463.182863 0.099915 61.930466 338.230561
28 5.679532 333.989714 0.104689 59.789692 315.095227
29 5.882373 235.423124 0.105729 68.450745 292.016484
30 6.085213 181.619742 0.115239 57.373480 279.435521
31 6.288053 181.495616 0.108327 41.251592 249.895779
32 6.490894 148.909799 0.118839 35.085998 247.667502
33 6.693734 106.653516 0.119293 46.731617 239.819781
34 6.896575 84.814190 0.136066 36.333522 233.639313
35 7.099415 62.885857 0.143851 33.363703 225.104853
36 7.302256 49.142341 0.163903 29.187662 219.586332
37 7.505096 42.414318 0.167708 25.094932 211.677800
38 7.707937 35.731905 0.186967 22.673872 213.025897
39 7.910777 34.649072 0.178698 19.759611 207.721937
40 8.113617 31.643594 0.190582 17.731739 205.457445
41 8.316458 31.576549 0.186011 17.319263 204.370599
42 8.519298 29.711085 0.189580 12.962045 201.507173
43 8.722139 25.402004 0.208819 11.838999 204.054317
44 8.924979 25.926379 0.202578 12.193460 200.751330
45 9.127820 20.655846 0.203674 15.696196 202.049154
46 9.330660 21.501622 0.200420 17.031168 201.664938
47 9.533500 18.295158 0.209065 15.326751 199.929263
48 9.736341 17.630038 0.223855 12.849859 200.833431
49 9.939181 19.752946 0.192329 14.677334 197.869369
50 10.142022 16.063825 0.209530 12.870744 199.786499
51 10.344862 15.912455 0.199130 13.444717 198.187176
52 10.547703 16.142997 0.196508 12.938033 197.286536
53 10.750543 16.155689 0.202991 9.862055 196.469383
54 10.953383 13.920308 0.225402 9.658823 197.326058
55 11.156224 13.103081 0.203455 14.857868 197.065425
56 11.359064 12.809598 0.205613 15.158263 197.532912
57 11.561905 13.027732 0.190986 15.528232 195.861344
58 11.764745 13.853095 0.218252 8.374395 196.751609
59 11.967586 14.785452 0.180077 7.701431 193.496723
60 12.170426 10.671071 0.214098 15.013271 196.189601
61 12.373267 11.071802 0.232047 13.638011 195.897987
62 12.576107 10.001690 0.251982 10.366264 196.744730
63 12.778947 11.718612 0.199282 15.029773 194.662109

Inspecting the outcome of the fit

[18]:
ddm.fit_report(fit01, q_indices=[3,6,9,22], forced_qs=[4,16], use_new_tau=True, show=True)
[18]:
<xarray.Dataset>
Dimensions:          (parameter: 4, q: 64, lagtime: 40)
Coordinates:
  * parameter        (parameter) <U10 'Amplitude' 'Tau' 'Mu' 'Background'
  * q                (q) float64 0.0 0.2028 0.4057 0.6085 ... 12.37 12.58 12.78
  * lagtime          (lagtime) float64 0.02398 0.04796 0.07194 ... 12.59 14.36
Data variables:
    parameters       (parameter, q) float64 1.0 15.92 30.33 ... 196.7 194.7
    theory           (lagtime, q) float64 -0.1567 734.1 794.1 ... 206.7 206.4
    isf_data         (lagtime, q) float64 0.0 0.9997 0.9899 ... -0.6879 -6.08
    ddm_matrix_data  (lagtime, q) float64 0.0 295.8 314.5 ... 206.3 208.3 207.5
    A                (q) float64 -211.2 2.585e+05 1.024e+04 ... -1.699 -0.52
    B                float64 211.2
Attributes: (12/18)
    model:                          DDM Matrix - Polydisperse
    data_to_use:                    DDM Matrix
    initial_params_dict:            ["{'n': 0, 'value': 100.0, 'limits': [1.0...
    effective_diffusion_coeff:      0.7148238989957726
    tau_vs_q_slope:                 [-1.784258]
    msd_alpha:                      [1.12091413]
    ...                             ...
    DataDirectory:                  C:/Users/rmcgorty/Documents/GitHub/DDM-at...
    FileName:                       images_nobin_40x_128x128_8bit.tif
    pixel_size:                     0.242
    frame_rate:                     41.7
    BackgroundMethod:               0
    OverlapMethod:                  1
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_19_1.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_19_2.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_19_3.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_19_4.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_19_5.png
[32]:
########################################
# Plotting mu vs q                     #
########################################

plt.figure()
plt.semilogx(fit01.q, fit01.parameters.loc['Mu'], 'o', color='tab:red', ms=5)
plt.xlabel("$q$ ($\mu m^{-1}$)")
plt.ylabel("$\mu$ (s$^{-2}$)")


########################################
# Plotting mu vs q                     #
########################################

plt.figure()
plt.semilogy(fit01.q, fit01.parameters.loc['Mu'] * fit01.parameters.loc['Tau']**2, 'o', color='tab:brown', ms=5)
plt.xlabel("$q$ ($\mu m^{-1}$)")
plt.ylabel("$\mu \\tau^2$")
[32]:
Text(0, 0.5, '$\\mu \\tau^2$')
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_20_1.png
_images/Walkthrough_-_creating_a_new_DDM_fitting_function_20_2.png
[ ]: