Unfortunately, I don't believe there is a simple formula to arrive at the value for $F(X,Y)$ but I can give you a process that can get to the value without too much calculation. There are ${X+k-1 \choose k}$ numbers with $X$ $1's$ and $k$ $0's$. The reason why this is the case is that there are $X+k$ binary digits and the left leading $1$ already uses a slot. So there are $X+k-1$ digit slots remaining and we need to pick $k$ $0's$ to go in those slots, the rest are $1's$. In order to find the $Y$ position, we need to sum up all of the previous numbers of smaller sizes in the column. So we sum ${X-1 \choose 0}+{X \choose 1}+{X+1 \choose 2}+{X+2 \choose 3}+...+{X+k-1 \choose k}$. Conveniently this is just ${X+k \choose k}$. When $(X+k)$ is the digit size of $F(X,Y)$, the value for $k$ is the smallest number when ${X+k \choose k}\ge Y$ is true. Finding this value without trial and error is very difficult but with some upper and lower bounds, the number of trials can be reduced. $$\frac{(x+LB)!}{x!LB!}<\frac{(x+LB)^x}{x!}<\frac{(x+LB)^x}{(2\pi x)^{\frac{1}{2}}\left(\frac{x}{e}\right)^{x}e^{\frac{1}{12x+1}}}<Y$$ There are four expressions in the inequality above. The first expression from the left is the ${x+k \choose k}$ displayed in factorial form with $k$ being replaced with $LB$ (lower bound). The second expression from the left is the first expression but the $\frac{(x+k)!}{LB!}$ is replaced with a larger $(x+k)^x$ term, that is easier to isolate $LB$. The third expression from the left is the denominator of the second expression replaced with Stirling's approximation. There are two approximations, one is larger than the factorial term and one is smaller. The smaller one was used on the lower bound inequality because the smaller the denominator the larger the fraction. The fourth expression is just $Y$. Now I can use algebra on the last two expressions to find $LB$. $$(x+LB)^x<Y(2\pi x)^{\frac{1}{2}}\left(\frac{x}{e}\right)^{x}e^{\frac{1}{12x+1}}$$ $$(x+LB)<Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2+x}}$$ $$(x+LB)<Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2+x}}$$ $$LB<Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2+x}}-x$$
LB has to be a whole number and must be less than the expression on the right directly above so
$$LB=\lfloor Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2+x}}-x\rfloor$$
A very similar process can be followed to get an upper bound.
$$\frac{(x+UB)!}{x!UB!}>\frac{(UB+1)^x}{x!}>\frac{(UB+1)^x}{(2\pi x)^{\frac{1}{2}}\left(\frac{x}{e}\right)^{x}e^{\frac{1}{12x}}}>Y$$
$$(UB+1)^x>Y(2\pi x)^{\frac{1}{2}}\left(\frac{x}{e}\right)^{x}e^{\frac{1}{12x}}$$
$$(UB+1)>Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2}}$$
$$UB>Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2}}-1$$
$$UB=\lceil Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2}}-1\rceil$$
So now there is a range for $k$
$$\lceil Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2}}-1\rceil\ge k\ge \lfloor Y^{\frac{1}{x}}(2\pi x)^{\frac{1}{2x}}\left(\frac{x}{e}\right)e^{\frac{1}{12x^2+x}}-x\rfloor$$
The next part uses a recursive formula that indicates digit by digit if it is a one or a zero. The first input of the formula $G({a \choose b-1},{a \choose b}, c)$ is the number of numbers that could fill in the remaining empty slots when the left most unfilled digit is a zero. The second input of the formula is the number of numbers that could fill in the remaining empty slots when the left most unfilled digit is a one. The third number is how many more numbers you have to go down the column in order to reach $(X,Y)$. There are three possibilities. The first possibility is the third input is less than the first input $({a \choose b-1} > c)$. The left-most unfilled digit is a $0$ then the inputs of the function go from $G({a \choose b-1},{a \choose b}, c)$ to $G({a-1 \choose b-2},{a-2 \choose b-1}, c)$. The second possibility is the third input is equal to the first input $({a \choose b-1} = c)$.The left most unfilled digit is a $0$. Then the remaining unused $1’s$ are put in from left to right, then the remaining $0’s$. In this possibility we have our number and we are done. The third possibility is the third input is greater than the first input. The left most unfilled digit is a $1$ and the input of the function goes from $G({a \choose b-1},{a \choose b},c)$ to $G({a-1 \choose b-1},{a-1 \choose b},c-{a \choose b-1})$. The starting input is $G({x+k-1\choose k-1},{x+k-1\choose k},Y)$.
Let’s do an example $F(6,1000000)$. Let’s use the lower bound formula to find $k$. $$\lfloor (1000000)^{(\frac{1}{6})}*(12\pi)^{(\frac{1}{12})}*(\frac{6}{e})*e^{\frac{1}{12*36}}\rfloor=23$$
$${29 \choose 23}=475020$$ $${30 \choose 24}=593775$$ $${31 \choose 25}=736281$$ $${32 \choose 26}=906192$$ $${33 \choose 27}=1107568$$
$$k=27$$
The function starts with $G({32 \choose 26},{32 \choose 27},1000000)$
$$\begin{array}{|c|c|c|c|c|} \hline function&input\space 1&input\space 2&input\space 3\space is >,<, or = input\space 1&filled\space in\space digits\\ \hline G({32 \choose 26},{32 \choose 27},1000000)&906192&201376&>&1\\ \hline G({31 \choose 26},{31 \choose 27},93808)&169911&31465&<&10\\ \hline G({30 \choose 25},{30 \choose 26},93808)&142506&27405&<&100\\ \hline G({29 \choose 24},{29 \choose 25},93808)&118755&23751&<&1000\\ \hline G({28 \choose 23},{28 \choose 24},93808)&98280&20475&<&10000\\ \hline G({27 \choose 22},{27 \choose 23},93808)&80730&17550&>&100001\\ \hline G({26 \choose 22},{26 \choose 23},13078)&14950&2600&<&1000010\\ \hline G({25 \choose 21},{25 \choose 22},13078)&12650&2300&>&10000101\\ \hline G({24 \choose 21},{24 \choose 22},428)&2024&276&<&100001010\\ \hline G({23 \choose 20},{23 \choose 21},428)&1771&253&<&1000010100\\ \hline G({22 \choose 19},{22 \choose 20},428)&1540&231&<&10000101000\\ \hline G({21 \choose 18},{21 \choose 19},428)&1330&210&<&100001010000\\ \hline G({20 \choose 17},{20 \choose 18},428)&1140&190&<&1000010100000\\ \hline G({19 \choose 16},{19 \choose 17},428)&969&171&<&10000101000000\\ \hline G({18 \choose 15},{18 \choose 16},428)&816&153&<&100001010000000\\ \hline G({17 \choose 14},{17 \choose 15},428)&680&136&<&1000010100000000\\ \hline G({16 \choose 13},{16 \choose 14},428)&560&120&<&10000101000000000\\ \hline G({15 \choose 12},{15 \choose 13},428)&455&105&<&100001010000000000\\ \hline G({14 \choose 11},{14 \choose 12},428)&364&91&>&1000010100000000001\\ \hline G({13 \choose 11},{13 \choose 12},64)&78&13&<&10000101000000000010\\ \hline G({12 \choose 10},{12 \choose 11},64)&66&12&<&100001010000000000100\\ \hline G({11 \choose 9},{11 \choose 10},64)&55&11&>&1000010100000000001001\\ \hline G({10 \choose 9},{10 \choose 10},9)&10&1&<&10000101000000000010010\\ \hline G({9 \choose 8},{9 \choose 9},9)&9&1&=&100001010000000000100100100000000\\ \hline \end{array}$$
$F(6,1000000)=2^{32}+2^{27}+2^{25}+2^{14}+2^{11}+2^8=4462758144$
