[Csnd-dev] Array corrections

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Csnd-dev] Array corrections

moguillansky

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op


Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

> On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:
>
> Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.
>
> OSClisten              kk[]                            iSS
> OSCraw                 S[]k                            i
> abs                    k[]                             k[]
> bformdec1              a[]                             ia[]
> bformenc1              a[]                             akk
> bpf                    k[]                             k[]M
> bpfcos                 k[]                             k[]M
> c2r                    k[]                             k[]
> cbrt                   k[]                             k[]
> ceil                   k[]                             k[]
> ceps                   k[]                             k[]k
> cepsinv                k[]                             k[]
> cmp                    k[]                             kSk[]Sk
> cmp                    k[]                             k[]Sk
> cmp                    k[]                             k[]Sk[]
> cmplxprod              k[]                             k[]k[]
> cos                    k[]                             k[]
> cosh                   k[]                             k[]
> cosinv                 k[]                             k[]
> dct                    k[]                             k[]
> dctinv                 k[]                             k[]
> deinterleave           k[]k[]                          k[]
> directory              S[]                             SN
> diskin                 a[]                             SPooooooo
> diskin                 a[]                             iPooooooo
> diskin2                a[]                             SPooooooo
> diskin2                a[]                             iPooooooo
> exp                    k[]                             k[]
> fft                    k[]                             k[]
> fftinv                 k[]                             k[]
> fillarray              S[]                             W
> fillarray              k[]                             S
> fillarray              k[]                             m
> fillarray              k[]                             z
> floor                  k[]                             k[]
> fluidInfo              S[]                             i
> fmax                   k[]                             k[]k
> fmax                   k[]                             k[]k[]
> fmin                   k[]                             k[]k
> fmin                   k[]                             k[]k[]
> fmod                   k[]                             k[]k
> fmod                   k[]                             k[]k[]
> frac                   k[]                             k[]
> ftom                   k[]                             k[]o
> genarray               k[]                             kkp
> genarray_i             k[]                             iip
> getcol                 k[]                             k[]k
> getrow                 k[]                             k[]k
> getrowlin              k[]                             k[]kOOP
> getrowlin              k[]                             kiiooop
> hypot                  k[]                             k[]k[]
> in                     a[]                             (null)
> in                     a[]                             (null)
> init                   .[]                             m
> inletv                 a[]                             S
> int                    k[]                             k[]
> interleave             k[]                             k[]k[]
> limit                  k[]                             k[]kk
> limit1                 k[]                             k[]
> linlin                 k[]                             k[]kkOP
> linlin                 k[]                             kk[]k[]OP
> log                    k[]                             k[]
> log                    k[]                             k[]i
> log10                  k[]                             k[]
> log2                   k[]                             k[]
> loscilx                a[]                             xkioojjoo
> mags                   k[]                             k[]
> maparray               k[]                             k[]S
> mfb                    k[]                             k[]kki
> monitor                a[]                             (null)
> mtof                   k[]                             k[]
> passign                k[]                             po
> phs                    k[]                             k[]
> pol2rect               k[]                             k[]
> pol2rect               k[]                             k[]k[]
> pow                    k[]                             k[]k
> pow                    k[]                             k[]k[]
> powoftwo               k[]                             k[]
> pows                   k[]                             k[]
> pvsceps                k[]                             fo
> pvstrace               fk[]                            fko
> r2c                    k[]                             k[]
> rect2pol               k[]                             k[]
> rfft                   k[]                             k[]
> rifft                  k[]                             k[]
> round                  k[]                             k[]
> setcol                 k[]                             k[]k
> setrow                 k[]                             k[]k
> shiftin                k[]                             a
> sin                    k[]                             k[]
> sinh                   k[]                             k[]
> sininv                 k[]                             k[]
> slicearray             S[]                             S[]iip
> slicearray             a[]                             a[]iip
> slicearray             k[]                             k[]iip
> sorta                  k[]                             k[]
> sortd                  k[]                             k[]
> sqrt                   k[]                             k[]
> string2array           k[]                             S
> tab2array              k[]                             iOOP
> tan                    k[]                             k[]
> tanh                   k[]                             k[]
> taninv                 k[]                             k[]
> taninv                 k[]                             k[]k[]
> unwrap                 k[]                             k[]
> vbap                   a[]                             akOOo
> vbapg                  k[]                             kOOo
> vbapgmove              k[]                             iiim
> vbapmove               a[]                             aiiim
> window                 k[]                             k[]Op
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

moguillansky
If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op




    
Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op




Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Steven Yi
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op




Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
As long as calloc and friends are not called at perf time, I am happy with any fix you feel
is the right one.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 23:14, Steven Yi <[hidden email]> wrote:

The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op




Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

moguillansky
In reply to this post by Steven Yi
Given the requirement that there can't be allocation at perf-time, the semantics for arrays should be:

* an array has an allocated size and a perceived size
* the allocated size is given at init time and can't be modified after that
* the output of any array operation should either result in an array of the expected size or in an error. This needs to be enfor
* for element wise operations, like arithmetic between an array and a scalar, the output array should be the same size / shape as the input array, or result in a performance error if this can't be achieved

If there is no agreement in the semantics, no code can be correct.

There are two issues here:

1) array growth. With the implementation in the array-init-change branch, arrays can be reallocated at perf-time if they need to grow beyond allocated size. According to Victor this needs to be prevented in order to ensure realtime safety. That means that tabensure, as written there, can't be called at perf-time.
2) correct perceived size. At perf-time it must be ensured that an array has the correct perceived size, as long as this size is <= allocated size. This operation does not imply any memory allocation.

With my proposed implementation all these constraints are satisfied. Any other solution is either semantically flawed (the currect situation, where no update of perceived size is done, size depends on allocated size, even on previous size in previous instances of the instrument, etc) or does not meet the required conditions (realtime safety)


On 26.04.19 00:13, Steven Yi wrote:
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op



Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Steven Yi
I was thinking through this and realized even if we say array sizes can only change at init-time, we may still need to set sizes at runtime.  It dawned on me that the situation occurs if you reuse an array var:

karr[] fillarray 1,2,3
...1. do something...
karr = fillarray 5,6,7,8
... 2. do something ...

so that in code section 1, karr should have a size of 3 at runtime, while code section 2, karr would have a size of 4. 

So it seems back to two tabensures, one at init and one for perf. 





On Thu, Apr 25, 2019 at 6:41 PM Eduardo Moguillansky <[hidden email]> wrote:
Given the requirement that there can't be allocation at perf-time, the semantics for arrays should be:

* an array has an allocated size and a perceived size
* the allocated size is given at init time and can't be modified after that
* the output of any array operation should either result in an array of the expected size or in an error. This needs to be enfor
* for element wise operations, like arithmetic between an array and a scalar, the output array should be the same size / shape as the input array, or result in a performance error if this can't be achieved

If there is no agreement in the semantics, no code can be correct.

There are two issues here:

1) array growth. With the implementation in the array-init-change branch, arrays can be reallocated at perf-time if they need to grow beyond allocated size. According to Victor this needs to be prevented in order to ensure realtime safety. That means that tabensure, as written there, can't be called at perf-time.
2) correct perceived size. At perf-time it must be ensured that an array has the correct perceived size, as long as this size is <= allocated size. This operation does not imply any memory allocation.

With my proposed implementation all these constraints are satisfied. Any other solution is either semantically flawed (the currect situation, where no update of perceived size is done, size depends on allocated size, even on previous size in previous instances of the instrument, etc) or does not meet the required conditions (realtime safety)


On 26.04.19 00:13, Steven Yi wrote:
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op



Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Mauro Giubileo
In reply to this post by Victor Lazzarini-2

I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?

---
Mauro


Il 2019-04-25 21:50 Victor Lazzarini ha scritto:

I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op


Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.

In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc. 

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:

I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?

---
Mauro


Il 2019-04-25 21:50 Victor Lazzarini ha scritto:

I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op


Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Mauro Giubileo

Thanks for the explanation. But why memory allocations have to be made on a separated thread? It's a Csound thing or it is related to the Bela platform?

---
Mauro


Il 2019-04-26 17:17 Victor Lazzarini ha scritto:

In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.
 
In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc. 

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:

I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?

---
Mauro


Il 2019-04-25 21:50 Victor Lazzarini ha scritto:

I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op

Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
It's a realtime safe thing. Any system that is hard realtime low-latency needs that. Csound offers it in its --realtime mode.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 19:12, Mauro Giubileo <[hidden email]> wrote:

Thanks for the explanation. But why memory allocations have to be made on a separated thread? It's a Csound thing or it is related to the Bela platform?

---
Mauro


Il 2019-04-26 17:17 Victor Lazzarini ha scritto:

In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.
 
In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc. 

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:

I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?

---
Mauro


Il 2019-04-25 21:50 Victor Lazzarini ha scritto:

I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op

Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Mauro Giubileo

Ok, so the separated thread is to make the audio processing run without interruptions from init-tasks of other instruments instances (I suppose) ?

---
Mauro


Il 2019-04-26 20:15 Victor Lazzarini ha scritto:

It's a realtime safe thing. Any system that is hard realtime low-latency needs that. Csound offers it in its --realtime mode.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 19:12, Mauro Giubileo <[hidden email]> wrote:

Thanks for the explanation. But why memory allocations have to be made on a separated thread? It's a Csound thing or it is related to the Bela platform?

---
Mauro


Il 2019-04-26 17:17 Victor Lazzarini ha scritto:

In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.
 
In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc. 

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:

I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?

---
Mauro


Il 2019-04-25 21:50 Victor Lazzarini ha scritto:

I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op
Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Justin Smith
It's about not getting blocked by syscalls. When you allocate memory
from the OS there's no way to guarantee it happens within a specific
period of time, so an app that is time critical should do memory
allocation either before time critical operations start, or in a
separate thread that isn't time critical.

On Fri, Apr 26, 2019 at 11:53 AM Mauro Giubileo
<[hidden email]> wrote:

>
> Ok, so the separated thread is to make the audio processing run without interruptions from init-tasks of other instruments instances (I suppose) ?
>
> ---
> Mauro
>
>
> Il 2019-04-26 20:15 Victor Lazzarini ha scritto:
>
> It's a realtime safe thing. Any system that is hard realtime low-latency needs that. Csound offers it in its --realtime mode.
>
> Victor Lazzarini
> Dean of Arts, Celtic Studies, and Philosophy
> Maynooth University
> Ireland
>
> On 26 Apr 2019, at 19:12, Mauro Giubileo <[hidden email]> wrote:
>
> Thanks for the explanation. But why memory allocations have to be made on a separated thread? It's a Csound thing or it is related to the Bela platform?
>
> ---
> Mauro
>
>
> Il 2019-04-26 17:17 Victor Lazzarini ha scritto:
>
> In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.
>
> In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc.
>
> Victor Lazzarini
> Dean of Arts, Celtic Studies, and Philosophy
> Maynooth University
> Ireland
>
> On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:
>
> I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?
>
> ---
> Mauro
>
>
> Il 2019-04-25 21:50 Victor Lazzarini ha scritto:
>
> I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
> calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
> operation where memory allocation can only happen at init-time on a separate thread.
>
> If this is a solution to a problem, then it's not the right solution.
> ========================
> Prof. Victor Lazzarini
> Dean of Arts, Celtic Studies, and Philosophy,
> Maynooth University,
> Maynooth, Co Kildare, Ireland
> Tel: 00 353 7086936
> Fax: 00 353 1 7086952
>
> On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:
>
> Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.
>
> OSClisten              kk[]                            iSS
> OSCraw                 S[]k                            i
> abs                    k[]                             k[]
> bformdec1              a[]                             ia[]
> bformenc1              a[]                             akk
> bpf                    k[]                             k[]M
> bpfcos                 k[]                             k[]M
> c2r                    k[]                             k[]
> cbrt                   k[]                             k[]
> ceil                   k[]                             k[]
> ceps                   k[]                             k[]k
> cepsinv                k[]                             k[]
> cmp                    k[]                             kSk[]Sk
> cmp                    k[]                             k[]Sk
> cmp                    k[]                             k[]Sk[]
> cmplxprod              k[]                             k[]k[]
> cos                    k[]                             k[]
> cosh                   k[]                             k[]
> cosinv                 k[]                             k[]
> dct                    k[]                             k[]
> dctinv                 k[]                             k[]
> deinterleave           k[]k[]                          k[]
> directory              S[]                             SN
> diskin                 a[]                             SPooooooo
> diskin                 a[]                             iPooooooo
> diskin2                a[]                             SPooooooo
> diskin2                a[]                             iPooooooo
> exp                    k[]                             k[]
> fft                    k[]                             k[]
> fftinv                 k[]                             k[]
> fillarray              S[]                             W
> fillarray              k[]                             S
> fillarray              k[]                             m
> fillarray              k[]                             z
> floor                  k[]                             k[]
> fluidInfo              S[]                             i
> fmax                   k[]                             k[]k
> fmax                   k[]                             k[]k[]
> fmin                   k[]                             k[]k
> fmin                   k[]                             k[]k[]
> fmod                   k[]                             k[]k
> fmod                   k[]                             k[]k[]
> frac                   k[]                             k[]
> ftom                   k[]                             k[]o
> genarray               k[]                             kkp
> genarray_i             k[]                             iip
> getcol                 k[]                             k[]k
> getrow                 k[]                             k[]k
> getrowlin              k[]                             k[]kOOP
> getrowlin              k[]                             kiiooop
> hypot                  k[]                             k[]k[]
> in                     a[]                             (null)
> in                     a[]                             (null)
> init                   .[]                             m
> inletv                 a[]                             S
> int                    k[]                             k[]
> interleave             k[]                             k[]k[]
> limit                  k[]                             k[]kk
> limit1                 k[]                             k[]
> linlin                 k[]                             k[]kkOP
> linlin                 k[]                             kk[]k[]OP
> log                    k[]                             k[]
> log                    k[]                             k[]i
> log10                  k[]                             k[]
> log2                   k[]                             k[]
> loscilx                a[]                             xkioojjoo
> mags                   k[]                             k[]
> maparray               k[]                             k[]S
> mfb                    k[]                             k[]kki
> monitor                a[]                             (null)
> mtof                   k[]                             k[]
> passign                k[]                             po
> phs                    k[]                             k[]
> pol2rect               k[]                             k[]
> pol2rect               k[]                             k[]k[]
> pow                    k[]                             k[]k
> pow                    k[]                             k[]k[]
> powoftwo               k[]                             k[]
> pows                   k[]                             k[]
> pvsceps                k[]                             fo
> pvstrace               fk[]                            fko
> r2c                    k[]                             k[]
> rect2pol               k[]                             k[]
> rfft                   k[]                             k[]
> rifft                  k[]                             k[]
> round                  k[]                             k[]
> setcol                 k[]                             k[]k
> setrow                 k[]                             k[]k
> shiftin                k[]                             a
> sin                    k[]                             k[]
> sinh                   k[]                             k[]
> sininv                 k[]                             k[]
> slicearray             S[]                             S[]iip
> slicearray             a[]                             a[]iip
> slicearray             k[]                             k[]iip
> sorta                  k[]                             k[]
> sortd                  k[]                             k[]
> sqrt                   k[]                             k[]
> string2array           k[]                             S
> tab2array              k[]                             iOOP
> tan                    k[]                             k[]
> tanh                   k[]                             k[]
> taninv                 k[]                             k[]
> taninv                 k[]                             k[]k[]
> unwrap                 k[]                             k[]
> vbap                   a[]                             akOOo
> vbapg                  k[]                             kOOo
> vbapgmove              k[]                             iiim
> vbapmove               a[]                             aiiim
> window                 k[]                             k[]Op
Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
That, and in the case of Csound, opening files, and also making blocking calls. All of that goes to separate threads that run in parallel with audio computing in --realtime
mode. In normal operation this all happens in the same thread.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

> On 26 Apr 2019, at 19:59, Justin Smith <[hidden email]> wrote:
>
> It's about not getting blocked by syscalls. When you allocate memory
> from the OS there's no way to guarantee it happens within a specific
> period of time, so an app that is time critical should do memory
> allocation either before time critical operations start, or in a
> separate thread that isn't time critical.
>
> On Fri, Apr 26, 2019 at 11:53 AM Mauro Giubileo
> <[hidden email]> wrote:
>>
>> Ok, so the separated thread is to make the audio processing run without interruptions from init-tasks of other instruments instances (I suppose) ?
>>
>> ---
>> Mauro
>>
>>
>> Il 2019-04-26 20:15 Victor Lazzarini ha scritto:
>>
>> It's a realtime safe thing. Any system that is hard realtime low-latency needs that. Csound offers it in its --realtime mode.
>>
>> Victor Lazzarini
>> Dean of Arts, Celtic Studies, and Philosophy
>> Maynooth University
>> Ireland
>>
>> On 26 Apr 2019, at 19:12, Mauro Giubileo <[hidden email]> wrote:
>>
>> Thanks for the explanation. But why memory allocations have to be made on a separated thread? It's a Csound thing or it is related to the Bela platform?
>>
>> ---
>> Mauro
>>
>>
>> Il 2019-04-26 17:17 Victor Lazzarini ha scritto:
>>
>> In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.
>>
>> In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc.
>>
>> Victor Lazzarini
>> Dean of Arts, Celtic Studies, and Philosophy
>> Maynooth University
>> Ireland
>>
>> On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:
>>
>> I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?
>>
>> ---
>> Mauro
>>
>>
>> Il 2019-04-25 21:50 Victor Lazzarini ha scritto:
>>
>> I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
>> calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
>> operation where memory allocation can only happen at init-time on a separate thread.
>>
>> If this is a solution to a problem, then it's not the right solution.
>> ========================
>> Prof. Victor Lazzarini
>> Dean of Arts, Celtic Studies, and Philosophy,
>> Maynooth University,
>> Maynooth, Co Kildare, Ireland
>> Tel: 00 353 7086936
>> Fax: 00 353 1 7086952
>>
>> On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:
>>
>> Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.
>>
>> OSClisten              kk[]                            iSS
>> OSCraw                 S[]k                            i
>> abs                    k[]                             k[]
>> bformdec1              a[]                             ia[]
>> bformenc1              a[]                             akk
>> bpf                    k[]                             k[]M
>> bpfcos                 k[]                             k[]M
>> c2r                    k[]                             k[]
>> cbrt                   k[]                             k[]
>> ceil                   k[]                             k[]
>> ceps                   k[]                             k[]k
>> cepsinv                k[]                             k[]
>> cmp                    k[]                             kSk[]Sk
>> cmp                    k[]                             k[]Sk
>> cmp                    k[]                             k[]Sk[]
>> cmplxprod              k[]                             k[]k[]
>> cos                    k[]                             k[]
>> cosh                   k[]                             k[]
>> cosinv                 k[]                             k[]
>> dct                    k[]                             k[]
>> dctinv                 k[]                             k[]
>> deinterleave           k[]k[]                          k[]
>> directory              S[]                             SN
>> diskin                 a[]                             SPooooooo
>> diskin                 a[]                             iPooooooo
>> diskin2                a[]                             SPooooooo
>> diskin2                a[]                             iPooooooo
>> exp                    k[]                             k[]
>> fft                    k[]                             k[]
>> fftinv                 k[]                             k[]
>> fillarray              S[]                             W
>> fillarray              k[]                             S
>> fillarray              k[]                             m
>> fillarray              k[]                             z
>> floor                  k[]                             k[]
>> fluidInfo              S[]                             i
>> fmax                   k[]                             k[]k
>> fmax                   k[]                             k[]k[]
>> fmin                   k[]                             k[]k
>> fmin                   k[]                             k[]k[]
>> fmod                   k[]                             k[]k
>> fmod                   k[]                             k[]k[]
>> frac                   k[]                             k[]
>> ftom                   k[]                             k[]o
>> genarray               k[]                             kkp
>> genarray_i             k[]                             iip
>> getcol                 k[]                             k[]k
>> getrow                 k[]                             k[]k
>> getrowlin              k[]                             k[]kOOP
>> getrowlin              k[]                             kiiooop
>> hypot                  k[]                             k[]k[]
>> in                     a[]                             (null)
>> in                     a[]                             (null)
>> init                   .[]                             m
>> inletv                 a[]                             S
>> int                    k[]                             k[]
>> interleave             k[]                             k[]k[]
>> limit                  k[]                             k[]kk
>> limit1                 k[]                             k[]
>> linlin                 k[]                             k[]kkOP
>> linlin                 k[]                             kk[]k[]OP
>> log                    k[]                             k[]
>> log                    k[]                             k[]i
>> log10                  k[]                             k[]
>> log2                   k[]                             k[]
>> loscilx                a[]                             xkioojjoo
>> mags                   k[]                             k[]
>> maparray               k[]                             k[]S
>> mfb                    k[]                             k[]kki
>> monitor                a[]                             (null)
>> mtof                   k[]                             k[]
>> passign                k[]                             po
>> phs                    k[]                             k[]
>> pol2rect               k[]                             k[]
>> pol2rect               k[]                             k[]k[]
>> pow                    k[]                             k[]k
>> pow                    k[]                             k[]k[]
>> powoftwo               k[]                             k[]
>> pows                   k[]                             k[]
>> pvsceps                k[]                             fo
>> pvstrace               fk[]                            fko
>> r2c                    k[]                             k[]
>> rect2pol               k[]                             k[]
>> rfft                   k[]                             k[]
>> rifft                  k[]                             k[]
>> round                  k[]                             k[]
>> setcol                 k[]                             k[]k
>> setrow                 k[]                             k[]k
>> shiftin                k[]                             a
>> sin                    k[]                             k[]
>> sinh                   k[]                             k[]
>> sininv                 k[]                             k[]
>> slicearray             S[]                             S[]iip
>> slicearray             a[]                             a[]iip
>> slicearray             k[]                             k[]iip
>> sorta                  k[]                             k[]
>> sortd                  k[]                             k[]
>> sqrt                   k[]                             k[]
>> string2array           k[]                             S
>> tab2array              k[]                             iOOP
>> tan                    k[]                             k[]
>> tanh                   k[]                             k[]
>> taninv                 k[]                             k[]
>> taninv                 k[]                             k[]k[]
>> unwrap                 k[]                             k[]
>> vbap                   a[]                             akOOo
>> vbapg                  k[]                             kOOo
>> vbapgmove              k[]                             iiim
>> vbapmove               a[]                             aiiim
>> window                 k[]                             k[]Op
Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Mauro Giubileo

Thank you very much, now everything is clearer to me.

---
Mauro


Il 2019-04-26 21:09 Victor Lazzarini ha scritto:

That, and in the case of Csound, opening files, and also making blocking calls. All of that goes to separate threads that run in parallel with audio computing in --realtime
mode. In normal operation this all happens in the same thread.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 19:59, Justin Smith <[hidden email]> wrote:

It's about not getting blocked by syscalls. When you allocate memory
from the OS there's no way to guarantee it happens within a specific
period of time, so an app that is time critical should do memory
allocation either before time critical operations start, or in a
separate thread that isn't time critical.

On Fri, Apr 26, 2019 at 11:53 AM Mauro Giubileo
<[hidden email]> wrote:

Ok, so the separated thread is to make the audio processing run without interruptions from init-tasks of other instruments instances (I suppose) ?

---
Mauro


Il 2019-04-26 20:15 Victor Lazzarini ha scritto:

It's a realtime safe thing. Any system that is hard realtime low-latency needs that. Csound offers it in its --realtime mode.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 19:12, Mauro Giubileo <[hidden email]> wrote:

Thanks for the explanation. But why memory allocations have to be made on a separated thread? It's a Csound thing or it is related to the Bela platform?

---
Mauro


Il 2019-04-26 17:17 Victor Lazzarini ha scritto:

In general, for realtime safe operation, all blocking operations and resource allocation should not be done in the audio processing thread.

In the Bela platform, in particular, this causes context switches, which in its hard-realtime OS will cause interruptions in the processing leading to clicks etc.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 26 Apr 2019, at 15:43, Mauro Giubileo <[hidden email]> wrote:

I don't know the Bela platform, but I'm curious... Exactly, why on the Bela platform you should not do a memory allocation at perf-time? What is different compared to a memory allocation made on a standard PC?

---
Mauro


Il 2019-04-25 21:50 Victor Lazzarini ha scritto:

I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952

On 25 Apr 2019, at 19:38, Eduardo Moguillansky <[hidden email]> wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added.

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op
Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Mauro Giubileo
In reply to this post by Steven Yi

If for realtime needs we don't want to allow memory allocation syscalls at perf-time, the second fillarray in your example should not be allowed. It could be allowed only if the elements of the second fillarray were less than those of the first.

I think that if an audio application needs to let an array grow dinamically at perf-time, one can always preallocate a bigger array during init-time, in anticipation of that growth...

---
Mauro


Il 2019-04-26 00:57 Steven Yi ha scritto:

 
I was thinking through this and realized even if we say array sizes can only change at init-time, we may still need to set sizes at runtime.  It dawned on me that the situation occurs if you reuse an array var:
 
karr[] fillarray 1,2,3
...1. do something...
karr = fillarray 5,6,7,8
... 2. do something ...
 
so that in code section 1, karr should have a size of 3 at runtime, while code section 2, karr would have a size of 4. 
 
So it seems back to two tabensures, one at init and one for perf. 
 
 



On Thu, Apr 25, 2019 at 6:41 PM Eduardo Moguillansky <[hidden email]> wrote:
Given the requirement that there can't be allocation at perf-time, the semantics for arrays should be:
 
* an array has an allocated size and a perceived size
* the allocated size is given at init time and can't be modified after that
* the output of any array operation should either result in an array of the expected size or in an error. This needs to be enfor
* for element wise operations, like arithmetic between an array and a scalar, the output array should be the same size / shape as the input array, or result in a performance error if this can't be achieved
 
If there is no agreement in the semantics, no code can be correct.
 
There are two issues here:

1) array growth. With the implementation in the array-init-change branch, arrays can be reallocated at perf-time if they need to grow beyond allocated size. According to Victor this needs to be prevented in order to ensure realtime safety. That means that tabensure, as written there, can't be called at perf-time.
2) correct perceived size. At perf-time it must be ensured that an array has the correct perceived size, as long as this size is <= allocated size. This operation does not imply any memory allocation.
 
With my proposed implementation all these constraints are satisfied. Any other solution is either semantically flawed (the currect situation, where no update of perceived size is done, size depends on allocated size, even on previous size in previous instances of the instrument, etc) or does not meet the required conditions (realtime safety)
 

On 26.04.19 00:13, Steven Yi wrote:
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:
If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it's not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op



Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Steven Yi
In reply to this post by Steven Yi
I'm wondering if we need three functions:

1. array_size_ensure_init : Does size check and resizing (w/mem allocation) but doesn't set size at init time
2. array_size_init : Does size check and resizing, sets size
3. array_size_perf: Does size check and sets size, no resizing and gives error if allocated size too small

#1 would be for something like slicearray version that runs at perf time. It'd have to ensure the size, but doesn't actually set the active size, since it doesn't do a calculation until run-time.

#2 would be for opcodes that run at init-time.

#3 would be for perf-time opcodes that need to set the size. The max size is allocated with #1 or #2 at init-time. 

Question: Do we need this?  I'm thinking it might be a kind of safety for the situation where people mix the order of init- and perf-time array processing opcodes.



On Thu, Apr 25, 2019 at 6:57 PM Steven Yi <[hidden email]> wrote:
I was thinking through this and realized even if we say array sizes can only change at init-time, we may still need to set sizes at runtime.  It dawned on me that the situation occurs if you reuse an array var:

karr[] fillarray 1,2,3
...1. do something...
karr = fillarray 5,6,7,8
... 2. do something ...

so that in code section 1, karr should have a size of 3 at runtime, while code section 2, karr would have a size of 4. 

So it seems back to two tabensures, one at init and one for perf. 





On Thu, Apr 25, 2019 at 6:41 PM Eduardo Moguillansky <[hidden email]> wrote:
Given the requirement that there can't be allocation at perf-time, the semantics for arrays should be:

* an array has an allocated size and a perceived size
* the allocated size is given at init time and can't be modified after that
* the output of any array operation should either result in an array of the expected size or in an error. This needs to be enfor
* for element wise operations, like arithmetic between an array and a scalar, the output array should be the same size / shape as the input array, or result in a performance error if this can't be achieved

If there is no agreement in the semantics, no code can be correct.

There are two issues here:

1) array growth. With the implementation in the array-init-change branch, arrays can be reallocated at perf-time if they need to grow beyond allocated size. According to Victor this needs to be prevented in order to ensure realtime safety. That means that tabensure, as written there, can't be called at perf-time.
2) correct perceived size. At perf-time it must be ensured that an array has the correct perceived size, as long as this size is <= allocated size. This operation does not imply any memory allocation.

With my proposed implementation all these constraints are satisfied. Any other solution is either semantically flawed (the currect situation, where no update of perceived size is done, size depends on allocated size, even on previous size in previous instances of the instrument, etc) or does not meet the required conditions (realtime safety)


On 26.04.19 00:13, Steven Yi wrote:
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op



Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

moguillansky
What happens to an opcode like kA[] = kB[] * kx, where the size of  kB is not known at init time? It needs to possibly allocate  an initial size at init time, but size depends on values set later. Should opcodes like this require preallocated arrays? If allocation at perftime is banned, then this seems the case.
On a side note,there are many other opcodes doing allocation at perftime, like string opcodes. Should these also require preallocation?

On 28.04.19 00:43, Steven Yi wrote:
I'm wondering if we need three functions:

1. array_size_ensure_init : Does size check and resizing (w/mem allocation) but doesn't set size at init time
2. array_size_init : Does size check and resizing, sets size
3. array_size_perf: Does size check and sets size, no resizing and gives error if allocated size too small

#1 would be for something like slicearray version that runs at perf time. It'd have to ensure the size, but doesn't actually set the active size, since it doesn't do a calculation until run-time.

#2 would be for opcodes that run at init-time.

#3 would be for perf-time opcodes that need to set the size. The max size is allocated with #1 or #2 at init-time. 

Question: Do we need this?  I'm thinking it might be a kind of safety for the situation where people mix the order of init- and perf-time array processing opcodes.



On Thu, Apr 25, 2019 at 6:57 PM Steven Yi <[hidden email]> wrote:
I was thinking through this and realized even if we say array sizes can only change at init-time, we may still need to set sizes at runtime.  It dawned on me that the situation occurs if you reuse an array var:

karr[] fillarray 1,2,3
...1. do something...
karr = fillarray 5,6,7,8
... 2. do something ...

so that in code section 1, karr should have a size of 3 at runtime, while code section 2, karr would have a size of 4. 

So it seems back to two tabensures, one at init and one for perf. 





On Thu, Apr 25, 2019 at 6:41 PM Eduardo Moguillansky <[hidden email]> wrote:
Given the requirement that there can't be allocation at perf-time, the semantics for arrays should be:

* an array has an allocated size and a perceived size
* the allocated size is given at init time and can't be modified after that
* the output of any array operation should either result in an array of the expected size or in an error. This needs to be enfor
* for element wise operations, like arithmetic between an array and a scalar, the output array should be the same size / shape as the input array, or result in a performance error if this can't be achieved

If there is no agreement in the semantics, no code can be correct.

There are two issues here:

1) array growth. With the implementation in the array-init-change branch, arrays can be reallocated at perf-time if they need to grow beyond allocated size. According to Victor this needs to be prevented in order to ensure realtime safety. That means that tabensure, as written there, can't be called at perf-time.
2) correct perceived size. At perf-time it must be ensured that an array has the correct perceived size, as long as this size is <= allocated size. This operation does not imply any memory allocation.

With my proposed implementation all these constraints are satisfied. Any other solution is either semantically flawed (the currect situation, where no update of perceived size is done, size depends on allocated size, even on previous size in previous instances of the instrument, etc) or does not meet the required conditions (realtime safety)


On 26.04.19 00:13, Steven Yi wrote:
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op



Reply | Threaded
Open this post in threaded view
|

Re: [Csnd-dev] Array corrections

Victor Lazzarini-2
In general, if kB does not have allocation at perf-time then an error should result, or
something of the kind.

String opcodes are a funny case that probably crept in with time, as originally
they only worked at i-time. We might need
to leave these alone, with a warning that
hard-realtime dsp use should avoid their
use. It's a different case in that it is not
really needed or used in signal processing.
Conversely, it's likely that users will want to use k-rate arrays in signal processing.

Off the top of my head, it appears that all array operations need to have an i-time 
pass so that any allocation needed happens
at that stage.

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 28 Apr 2019, at 09:36, Eduardo Moguillansky <[hidden email]> wrote:

What happens to an opcode like kA[] = kB[] * kx, where the size of  kB is not known at init time? It needs to possibly allocate  an initial size at init time, but size depends on values set later. Should opcodes like this require preallocated arrays? If allocation at perftime is banned, then this seems the case.
On a side note,there are many other opcodes doing allocation at perftime, like string opcodes. Should these also require preallocation?

On 28.04.19 00:43, Steven Yi wrote:
I'm wondering if we need three functions:

1. array_size_ensure_init : Does size check and resizing (w/mem allocation) but doesn't set size at init time
2. array_size_init : Does size check and resizing, sets size
3. array_size_perf: Does size check and sets size, no resizing and gives error if allocated size too small

#1 would be for something like slicearray version that runs at perf time. It'd have to ensure the size, but doesn't actually set the active size, since it doesn't do a calculation until run-time.

#2 would be for opcodes that run at init-time.

#3 would be for perf-time opcodes that need to set the size. The max size is allocated with #1 or #2 at init-time. 

Question: Do we need this?  I'm thinking it might be a kind of safety for the situation where people mix the order of init- and perf-time array processing opcodes.



On Thu, Apr 25, 2019 at 6:57 PM Steven Yi <[hidden email]> wrote:
I was thinking through this and realized even if we say array sizes can only change at init-time, we may still need to set sizes at runtime.  It dawned on me that the situation occurs if you reuse an array var:

karr[] fillarray 1,2,3
...1. do something...
karr = fillarray 5,6,7,8
... 2. do something ...

so that in code section 1, karr should have a size of 3 at runtime, while code section 2, karr would have a size of 4. 

So it seems back to two tabensures, one at init and one for perf. 





On Thu, Apr 25, 2019 at 6:41 PM Eduardo Moguillansky <[hidden email]> wrote:
Given the requirement that there can't be allocation at perf-time, the semantics for arrays should be:

* an array has an allocated size and a perceived size
* the allocated size is given at init time and can't be modified after that
* the output of any array operation should either result in an array of the expected size or in an error. This needs to be enfor
* for element wise operations, like arithmetic between an array and a scalar, the output array should be the same size / shape as the input array, or result in a performance error if this can't be achieved

If there is no agreement in the semantics, no code can be correct.

There are two issues here:

1) array growth. With the implementation in the array-init-change branch, arrays can be reallocated at perf-time if they need to grow beyond allocated size. According to Victor this needs to be prevented in order to ensure realtime safety. That means that tabensure, as written there, can't be called at perf-time.
2) correct perceived size. At perf-time it must be ensured that an array has the correct perceived size, as long as this size is <= allocated size. This operation does not imply any memory allocation.

With my proposed implementation all these constraints are satisfied. Any other solution is either semantically flawed (the currect situation, where no update of perceived size is done, size depends on allocated size, even on previous size in previous instances of the instrument, etc) or does not meet the required conditions (realtime safety)


On 26.04.19 00:13, Steven Yi wrote:
The code change for tabensure (the one in the array-init-change branch) seems good to me. If the problem is that it should not be called at perf-time, then existing opcodes have problems and need to be rewritten. I'd rather we not have tabensure_perf and just document "only use at init-time".  

On Thu, Apr 25, 2019 at 5:33 PM Victor Lazzarini <[hidden email]> wrote:
Sounds like the correct thing to me. As with other dynamic memory uses by opcodes, we only allocate or reallocate at init-time (e.g. we don't change delay line max sizes at perf time).

Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy
Maynooth University
Ireland

On 25 Apr 2019, at 22:21, Eduardo Moguillansky <[hidden email]> wrote:

If this is a requirement, then the array semantics need to be redefined and made clear. There can't be any operation at perf time which can grow an array. Even further, all perf time array operation need to operate with initialized arrays, since that would mean a perf time allocation. Following this semantics, we could go back to two tabensure functions in the form:


// this should be called at init time by any opcode returning an array 
// even if the opcode only works at perf-time.
static inline void tabensure_init(CSOUND *csound, ARRAYDAT *p, int size) {
    size_t ss = p->arrayMemberSize*size;
    if (p->data == NULL) {
        CS_VARIABLE* var = p->arrayType->createVariable(csound, NULL);
        p->arrayMemberSize = var->memBlockSize;
        p->data = (MYFLT*)csound->Calloc(csound, ss);
        p->allocated = ss;
    }
    else if (ss > p->allocated) {
        p->data = (MYFLT*)csound->ReAlloc(csound, p->data, ss);
        p->allocated = ss;
    }
    p->sizes[0] = size;
    p->dimensions = 1;
}

// this should be called at perf time
static inline void tabensure_perf(CSOUND *csound, ARRAYDAT *p, int size) {
    if(p->data == NULL) {
        return csound->PerformanceError(...);
    }
    size_t ss = p->arrayMemberSize*size;
    if(ss > p->allocated) {
        return csound->PerformanceError(...);
    }
    p->sizes[0] = size;
}

With this scheme, all allocation and resizing would happen at init time and 
any attempt to resize at perf time would result in an error. The user then
can preallocate arrays as big as needed:

kMyArray[] init i_largestsize possible

Would that be an acceptable solution?


On 25.04.19 21:50, Victor Lazzarini wrote:
I have been largely keeping out of this discussion, but this email is sounding alarm bells here. The tabensure() function calls
calloc() and that absolutely should be kept out of perf-time. It will break platforms (such as Bela) that require realtime safe
operation where memory allocation can only happen at init-time on a separate thread.

If this is a solution to a problem, then it’s not the right solution.
========================
Prof. Victor Lazzarini
Dean of Arts, Celtic Studies, and Philosophy,
Maynooth University,
Maynooth, Co Kildare, Ireland
Tel: 00 353 7086936
Fax: 00 353 1 7086952 

On 25 Apr 2019, at 19:38, Eduardo Moguillansky [hidden email] wrote:

Now that the array semantics seem clear (at least for 1D arrays) there is a big rewrite ahead comprising all opcodes which have arrays as output, which need to comply with this behaviour by calling tabensure at perf-time. The complete list of opcode which need to be revised is below (also all opcodes dealing with array arithmetic, which are not listed here). There are probably some of those which already do the right thing but most need to have the call to tabensure added. 

OSClisten              kk[]                            iSS
OSCraw                 S[]k                            i
abs                    k[]                             k[]
bformdec1              a[]                             ia[]
bformenc1              a[]                             akk
bpf                    k[]                             k[]M
bpfcos                 k[]                             k[]M
c2r                    k[]                             k[]
cbrt                   k[]                             k[]
ceil                   k[]                             k[]
ceps                   k[]                             k[]k
cepsinv                k[]                             k[]
cmp                    k[]                             kSk[]Sk
cmp                    k[]                             k[]Sk
cmp                    k[]                             k[]Sk[]
cmplxprod              k[]                             k[]k[]
cos                    k[]                             k[]
cosh                   k[]                             k[]
cosinv                 k[]                             k[]
dct                    k[]                             k[]
dctinv                 k[]                             k[]
deinterleave           k[]k[]                          k[]
directory              S[]                             SN
diskin                 a[]                             SPooooooo
diskin                 a[]                             iPooooooo
diskin2                a[]                             SPooooooo
diskin2                a[]                             iPooooooo
exp                    k[]                             k[]
fft                    k[]                             k[]
fftinv                 k[]                             k[]
fillarray              S[]                             W
fillarray              k[]                             S
fillarray              k[]                             m
fillarray              k[]                             z
floor                  k[]                             k[]
fluidInfo              S[]                             i
fmax                   k[]                             k[]k
fmax                   k[]                             k[]k[]
fmin                   k[]                             k[]k
fmin                   k[]                             k[]k[]
fmod                   k[]                             k[]k
fmod                   k[]                             k[]k[]
frac                   k[]                             k[]
ftom                   k[]                             k[]o
genarray               k[]                             kkp
genarray_i             k[]                             iip
getcol                 k[]                             k[]k
getrow                 k[]                             k[]k
getrowlin              k[]                             k[]kOOP
getrowlin              k[]                             kiiooop
hypot                  k[]                             k[]k[]
in                     a[]                             (null)
in                     a[]                             (null)
init                   .[]                             m
inletv                 a[]                             S
int                    k[]                             k[]
interleave             k[]                             k[]k[]
limit                  k[]                             k[]kk
limit1                 k[]                             k[]
linlin                 k[]                             k[]kkOP
linlin                 k[]                             kk[]k[]OP
log                    k[]                             k[]
log                    k[]                             k[]i
log10                  k[]                             k[]
log2                   k[]                             k[]
loscilx                a[]                             xkioojjoo
mags                   k[]                             k[]
maparray               k[]                             k[]S
mfb                    k[]                             k[]kki
monitor                a[]                             (null)
mtof                   k[]                             k[]
passign                k[]                             po
phs                    k[]                             k[]
pol2rect               k[]                             k[]
pol2rect               k[]                             k[]k[]
pow                    k[]                             k[]k
pow                    k[]                             k[]k[]
powoftwo               k[]                             k[]
pows                   k[]                             k[]
pvsceps                k[]                             fo
pvstrace               fk[]                            fko
r2c                    k[]                             k[]
rect2pol               k[]                             k[]
rfft                   k[]                             k[]
rifft                  k[]                             k[]
round                  k[]                             k[]
setcol                 k[]                             k[]k
setrow                 k[]                             k[]k
shiftin                k[]                             a
sin                    k[]                             k[]
sinh                   k[]                             k[]
sininv                 k[]                             k[]
slicearray             S[]                             S[]iip
slicearray             a[]                             a[]iip
slicearray             k[]                             k[]iip
sorta                  k[]                             k[]
sortd                  k[]                             k[]
sqrt                   k[]                             k[]
string2array           k[]                             S
tab2array              k[]                             iOOP
tan                    k[]                             k[]
tanh                   k[]                             k[]
taninv                 k[]                             k[]
taninv                 k[]                             k[]k[]
unwrap                 k[]                             k[]
vbap                   a[]                             akOOo
vbapg                  k[]                             kOOo
vbapgmove              k[]                             iiim
vbapmove               a[]                             aiiim
window                 k[]                             k[]Op



12